Mirror
/
mergerfs
mirror of https://github.com/trapexit/mergerfs.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

README: add terminology section and tweak some things

pull/758/head
Antonio SJ Musumeci 4 years ago
parent
a61d420224
commit
add588fdea
2 changed files with 50 additions and 48 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 32
      README.md
  2. 66
      man/mergerfs.1

32
README.md
View File

@ -1,6 +1,6 @@
% mergerfs(1) mergerfs user manual
% Antonio SJ Musumeci <trapexit@spawn.link>
% 2020-05-18
% 2020-05-25
# NAME
@ -62,6 +62,17 @@ A + B = C
mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives. It also does **not** split data across drives. It is not RAID0 / striping. It is simply a union.
# TERMINOLOGY
* branch: A base path used in the pool.
* pool: The mergerfs mount. The union of the branches.
* relative path: The path in the pool relative to the branch and mount.
* policy: The algorithm used to select a file when performing a function.
* function: A filesystem call (open, unlink, create, getattr, etc.)
* category: A collection of functions (action, create, search).
* path preservation: Aspect of some policies which includes checking the path for which a file would be created.
# BASIC SETUP
If you don't already know that you have a special use case then just start with one of the following option sets.
@ -757,7 +768,7 @@ If you want to move files to one drive just copy them there and use mergerfs.ded
#### cached memory appears greater than it should be
Use `cache.files=off` or `direct_io=true`. See the section on page caching.
Use `cache.files=off` and/or `dropcacheonclose=true`. See the section on page caching.
#### NFS clients returning ESTALE / Stale file handle
@ -905,11 +916,7 @@ There is a bug in the kernel. A work around appears to be turning off `splice`.
#### rm: fts_read failed: No such file or directory
NOTE: This is only relevant to mergerfs versions at or below v2.25.x and should not occur in more recent versions. See the notes on `unlink`.
Not *really* a bug. The FUSE library will move files when asked to delete them as a way to deal with certain edge cases and then later delete that file when its clear the file is no longer needed. This however can lead to two issues. One is that these hidden files are noticed by `rm -rf` or `find` when scanning directories and they may try to remove them and they might have disappeared already. There is nothing *wrong* about this happening but it can be annoying. The second issue is that a directory might not be able to removed on account of the hidden file being still there.
Using the **hard_remove** option will make it so these temporary files are not used and files are deleted immediately. That has a side effect however. Files which are unlinked and then they are still used (in certain forms) will result in an error (ENOENT).
Please update. This is only happened to mergerfs versions at or below v2.25.x and will not occur in more recent versions.
# FAQ
@ -1073,6 +1080,11 @@ It is also possible that the filesystem selected has run out of inodes. Use `df
If you don't care about path preservation then simply change the `create` policy to one which isn't. `mfs` is probably what most are looking for. The reason its not default is because it was originally set to `epmfs` and changing it now would change people's setup. Such a setting change will likely occur in mergerfs 3.
#### Why does the total available space in mergerfs not equal outside?
Are you using ext4? With reserve for root? mergerfs uses available space for statfs calculations. If you've reserved space for root then it won't show up.
#### Can mergerfs mounts be exported over NFS?
Yes. Due to current usage of libfuse by mergerfs and how NFS interacts with it it is necessary to add `noforget` to mergerfs options to keep from getting "stale file handle" errors.
@ -1123,11 +1135,7 @@ To work around this situation mergerfs offers a few solutions.
#### What are these .fuse_hidden files?
NOTE: mergerfs >= 2.26.0 will not have these temporary files. See the notes on `unlink`.
When not using **hard_remove** libfuse will create .fuse_hiddenXXXXXXXX files when an opened file is unlinked. This is to simplify "use after unlink" usecases. There is a possibility these files end up being picked up by software scanning directories and not ignoring hidden files. This is rarely a problem but a solution is in the works.
The files are cleaned up once the file is finally closed. Only if mergerfs crashes or is killed would they be left around. They are safe to remove as they are already unlinked files.
Please upgrade. mergerfs >= 2.26.0 will not have these temporary files. See the notes on `unlink`.
#### It's mentioned that there are some security issues with mhddfs. What are they? How does mergerfs address them?

66
man/mergerfs.1
View File

@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 1.19.2.4
.\"
.TH "mergerfs" "1" "2020\-05\-18" "mergerfs user manual" ""
.TH "mergerfs" "1" "2020\-05\-25" "mergerfs user manual" ""
.hy
.SH NAME
.PP
@ -82,6 +82,23 @@ so you can mix read\-write and read\-only drives.
It also does \f[B]not\f[] split data across drives.
It is not RAID0 / striping.
It is simply a union.
.SH TERMINOLOGY
.IP \[bu] 2
branch: A base path used in the pool.
.IP \[bu] 2
pool: The mergerfs mount.
The union of the branches.
.IP \[bu] 2
relative path: The path in the pool relative to the branch and mount.
.IP \[bu] 2
policy: The algorithm used to select a file when performing a function.
.IP \[bu] 2
function: A filesystem call (open, unlink, create, getattr, etc.)
.IP \[bu] 2
category: A collection of functions (action, create, search).
.IP \[bu] 2
path preservation: Aspect of some policies which includes checking the
path for which a file would be created.
.SH BASIC SETUP
.PP
If you don\[aq]t already know that you have a special use case then just
@ -1660,7 +1677,7 @@ mergerfs.dedup to clean up the old paths or manually remove them from
the branches directly.
.SS cached memory appears greater than it should be
.PP
Use \f[C]cache.files=off\f[] or \f[C]direct_io=true\f[].
Use \f[C]cache.files=off\f[] and/or \f[C]dropcacheonclose=true\f[].
See the section on page caching.
.SS NFS clients returning ESTALE / Stale file handle
.PP
@ -1920,28 +1937,9 @@ Don\[aq]t add the \f[C]splice_*\f[] arguments or add
This, however, is not guaranteed to work.
.SS rm: fts_read failed: No such file or directory
.PP
NOTE: This is only relevant to mergerfs versions at or below v2.25.x and
should not occur in more recent versions.
See the notes on \f[C]unlink\f[].
.PP
Not \f[I]really\f[] a bug.
The FUSE library will move files when asked to delete them as a way to
deal with certain edge cases and then later delete that file when its
clear the file is no longer needed.
This however can lead to two issues.
One is that these hidden files are noticed by \f[C]rm\ \-rf\f[] or
\f[C]find\f[] when scanning directories and they may try to remove them
and they might have disappeared already.
There is nothing \f[I]wrong\f[] about this happening but it can be
annoying.
The second issue is that a directory might not be able to removed on
account of the hidden file being still there.
.PP
Using the \f[B]hard_remove\f[] option will make it so these temporary
files are not used and files are deleted immediately.
That has a side effect however.
Files which are unlinked and then they are still used (in certain forms)
will result in an error (ENOENT).
Please update.
This is only happened to mergerfs versions at or below v2.25.x and will
not occur in more recent versions.
.SH FAQ
.SS How well does mergerfs scale? Is it "production ready?"
.PP
@ -2244,6 +2242,12 @@ If you don\[aq]t care about path preservation then simply change the
The reason its not default is because it was originally set to
\f[C]epmfs\f[] and changing it now would change people\[aq]s setup.
Such a setting change will likely occur in mergerfs 3.
.SS Why does the total available space in mergerfs not equal outside?
.PP
Are you using ext4?
With reserve for root?
mergerfs uses available space for statfs calculations.
If you\[aq]ve reserved space for root then it won\[aq]t show up.
.SS Can mergerfs mounts be exported over NFS?
.PP
Yes.
@ -2339,19 +2343,9 @@ probably simplier to just disable it all together.
The kernel won\[aq]t send the requests when caching is disabled.
.SS What are these .fuse_hidden files?
.PP
NOTE: mergerfs >= 2.26.0 will not have these temporary files.
Please upgrade.
mergerfs >= 2.26.0 will not have these temporary files.
See the notes on \f[C]unlink\f[].
.PP
When not using \f[B]hard_remove\f[] libfuse will create
\&.fuse_hiddenXXXXXXXX files when an opened file is unlinked.
This is to simplify "use after unlink" usecases.
There is a possibility these files end up being picked up by software
scanning directories and not ignoring hidden files.
This is rarely a problem but a solution is in the works.
.PP
The files are cleaned up once the file is finally closed.
Only if mergerfs crashes or is killed would they be left around.
They are safe to remove as they are already unlinked files.
.SS It\[aq]s mentioned that there are some security issues with mhddfs.
What are they? How does mergerfs address them?
.PP

Write Preview
Loading…
Cancel
Save