mirror of https://github.com/trapexit/mergerfs.git
Browse Source
Merge pull request #165 from trapexit/manpage
Merge pull request #165 from trapexit/manpage
remove manpage from root directorypull/167/head
Antonio SJ Musumeci
9 years ago
1 changed files with 0 additions and 759 deletions
-
759mergerfs.1
@ -1,759 +0,0 @@ |
|||
.\"t |
|||
.TH "mergerfs" "1" "2015\-10\-11" "mergerfs user manual" "" |
|||
.SH NAME |
|||
.PP |
|||
mergerfs \- another FUSE union filesystem |
|||
.SH SYNOPSIS |
|||
.PP |
|||
mergerfs \-o<options> <srcpoints> <mountpoint> |
|||
.SH DESCRIPTION |
|||
.PP |
|||
\f[B]mergerfs\f[] is similar to \f[B]mhddfs\f[], \f[B]unionfs\f[], and |
|||
\f[B]aufs\f[]. |
|||
Like \f[B]mhddfs\f[] in that it too uses \f[B]FUSE\f[]. |
|||
Like \f[B]aufs\f[] in that it provides multiple policies for how to |
|||
handle behavior. |
|||
.PP |
|||
Why \f[B]mergerfs\f[] when those exist? |
|||
\f[B]mhddfs\f[] has not been updated in some time nor very flexible. |
|||
There are also security issues when with running as root. |
|||
\f[B]aufs\f[] is more flexible than \f[B]mhddfs\f[] but kernel based and |
|||
difficult to debug when problems arise. |
|||
Neither support file attributes |
|||
(chattr (http://linux.die.net/man/1/chattr)). |
|||
.SH FEATURES |
|||
.IP \[bu] 2 |
|||
Runs in userspace (FUSE) |
|||
.IP \[bu] 2 |
|||
Configurable behaviors |
|||
.IP \[bu] 2 |
|||
Supports extended attributes (xattrs) |
|||
.IP \[bu] 2 |
|||
Supports file attributes (chattr) |
|||
.IP \[bu] 2 |
|||
Dynamically configurable (via xattrs) |
|||
.IP \[bu] 2 |
|||
Safe to run as root |
|||
.IP \[bu] 2 |
|||
Opportunistic credential caching |
|||
.IP \[bu] 2 |
|||
Works with heterogeneous filesystem types |
|||
.SH OPTIONS |
|||
.SS options |
|||
.IP \[bu] 2 |
|||
\f[B]defaults\f[]: a shortcut for FUSE\[aq]s \f[B]atomic_o_trunc\f[], |
|||
\f[B]auto_cache\f[], \f[B]big_writes\f[], \f[B]default_permissions\f[], |
|||
\f[B]splice_move\f[], \f[B]splice_read\f[], and \f[B]splice_write\f[]. |
|||
These options seem to provide the best performance. |
|||
.IP \[bu] 2 |
|||
\f[B]direct_io\f[]: causes FUSE to bypass an addition caching step which |
|||
can increase write speeds at the detriment of read speed. |
|||
.IP \[bu] 2 |
|||
\f[B]minfreespace\f[]: the minimum space value used for the |
|||
\f[B]lfs\f[], \f[B]fwfs\f[], and \f[B]epmfs\f[] policies. |
|||
Understands \[aq]K\[aq], \[aq]M\[aq], and \[aq]G\[aq] to represent |
|||
kilobyte, megabyte, and gigabyte respectively. |
|||
(default: 4G) |
|||
.IP \[bu] 2 |
|||
\f[B]moveonenospc\f[]: when enabled (set to \f[B]true\f[]) if a |
|||
\f[B]write\f[] fails with \f[B]ENOSPC\f[] a scan of all drives will be |
|||
done looking for the drive with most free space which is at least the |
|||
size of the file plus the amount which failed to write. |
|||
An attempt to move the file to that drive will occur (keeping all |
|||
metadata possible) and if successful the original is unlinked and the |
|||
write retried. |
|||
(default: false) |
|||
.IP \[bu] 2 |
|||
\f[B]func.<func>=<policy>\f[]: sets the specific FUSE function\[aq]s |
|||
policy. |
|||
See below for the list of value types. |
|||
Example: \f[B]func.getattr=newest\f[] |
|||
.IP \[bu] 2 |
|||
\f[B]category.<category>=<policy>\f[]: Sets policy of all FUSE functions |
|||
in the provided category. |
|||
Example: \f[B]category.create=mfs\f[] |
|||
.PP |
|||
\f[B]NOTE:\f[] Options are evaluated in the order listed so if the |
|||
options are \f[B]func.rmdir=rand,category.action=ff\f[] the |
|||
\f[B]action\f[] category setting will override the \f[B]rmdir\f[] |
|||
setting. |
|||
.SS srcpoints |
|||
.PP |
|||
The source points argument is a colon (\[aq]:\[aq]) delimited list of |
|||
paths. |
|||
To make it simpler to include multiple source points without having to |
|||
modify your fstab (http://linux.die.net/man/5/fstab) we also support |
|||
globbing (http://linux.die.net/man/7/glob). |
|||
\f[B]The globbing tokens MUST be escaped when using via the shell else |
|||
the shell itself will probably expand it.\f[] |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
$\ mergerfs\ /mnt/disk\\*:/mnt/cdrom\ /media/drives |
|||
\f[] |
|||
.fi |
|||
.PP |
|||
The above line will use all points in /mnt prefixed with \f[I]disk\f[] |
|||
and the directory \f[I]cdrom\f[]. |
|||
.PP |
|||
In /etc/fstab it\[aq]d look like the following: |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
#\ <file\ system>\ \ \ \ \ \ \ \ <mount\ point>\ \ <type>\ \ \ \ \ \ \ \ \ <options>\ \ \ \ \ \ \ \ \ \ \ \ \ <dump>\ \ <pass> |
|||
/mnt/disk*:/mnt/cdrom\ \ /media/drives\ \ fuse.mergerfs\ \ defaults,allow_other\ \ 0\ \ \ \ \ \ \ 0 |
|||
\f[] |
|||
.fi |
|||
.PP |
|||
\f[B]NOTE:\f[] the globbing is done at mount or xattr update time. |
|||
If a new directory is added matching the glob after the fact it will not |
|||
be included. |
|||
.SH POLICIES |
|||
.PP |
|||
Filesystem calls are broken up into 3 categories: \f[B]action\f[], |
|||
\f[B]create\f[], \f[B]search\f[]. |
|||
There are also some calls which have no policy attached due to state |
|||
being kept between calls. |
|||
These categories can be assigned a policy which dictates how |
|||
\f[B]mergerfs\f[] behaves. |
|||
Any policy can be assigned to a category though some aren\[aq]t terribly |
|||
practical. |
|||
For instance: \f[B]rand\f[] (Random) may be useful for \f[B]create\f[] |
|||
but could lead to very odd behavior if used for \f[B]search\f[]. |
|||
.SS Functional classifications |
|||
.PP |
|||
.TS |
|||
tab(@); |
|||
l l. |
|||
T{ |
|||
Category |
|||
T}@T{ |
|||
FUSE Functions |
|||
T} |
|||
_ |
|||
T{ |
|||
action |
|||
T}@T{ |
|||
chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, |
|||
unlink, utimens |
|||
T} |
|||
T{ |
|||
create |
|||
T}@T{ |
|||
create, mkdir, mknod, symlink |
|||
T} |
|||
T{ |
|||
search |
|||
T}@T{ |
|||
access, getattr, getxattr, ioctl, listxattr, open, readlink |
|||
T} |
|||
T{ |
|||
N/A |
|||
T}@T{ |
|||
fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, release, |
|||
statfs, write |
|||
T} |
|||
.TE |
|||
.PP |
|||
\f[B]ioctl\f[] behaves differently if its acting on a directory. |
|||
It\[aq]ll use the \f[B]getattr\f[] policy to find and open the directory |
|||
before issuing the \f[B]ioctl\f[]. |
|||
In other cases where something may be searched (to confirm a directory |
|||
exists across all source mounts) then \f[B]getattr\f[] will be used. |
|||
.SS Policy descriptions |
|||
.PP |
|||
.TS |
|||
tab(@); |
|||
l l. |
|||
T{ |
|||
Policy |
|||
T}@T{ |
|||
Description |
|||
T} |
|||
_ |
|||
T{ |
|||
ff (first found) |
|||
T}@T{ |
|||
Given the order of the drives act on the first one found (regardless if |
|||
stat would return EACCES). |
|||
T} |
|||
T{ |
|||
ffwp (first found w/ permissions) |
|||
T}@T{ |
|||
Given the order of the drives act on the first one found which you have |
|||
access (stat does not error with EACCES). |
|||
T} |
|||
T{ |
|||
newest (newest file) |
|||
T}@T{ |
|||
If multiple files exist return the one with the most recent mtime. |
|||
T} |
|||
T{ |
|||
mfs (most free space) |
|||
T}@T{ |
|||
Use the drive with the most free space available. |
|||
T} |
|||
T{ |
|||
epmfs (existing path, most free space) |
|||
T}@T{ |
|||
If the path exists on multiple drives use the one with the most free |
|||
space and is greater than \f[B]minfreespace\f[]. |
|||
If no drive has at least \f[B]minfreespace\f[] then fallback to |
|||
\f[B]mfs\f[]. |
|||
T} |
|||
T{ |
|||
fwfs (first with free space) |
|||
T}@T{ |
|||
Pick the first drive which has at least \f[B]minfreespace\f[]. |
|||
T} |
|||
T{ |
|||
lfs (least free space) |
|||
T}@T{ |
|||
Pick the drive with least available space but more than |
|||
\f[B]minfreespace\f[]. |
|||
T} |
|||
T{ |
|||
rand (random) |
|||
T}@T{ |
|||
Pick an existing drive at random. |
|||
T} |
|||
T{ |
|||
all |
|||
T}@T{ |
|||
Applies action to all found. |
|||
For searches it will behave like first found \f[B]ff\f[]. |
|||
T} |
|||
T{ |
|||
enosys, einval, enotsup, exdev, erofs |
|||
T}@T{ |
|||
Exclusively return \f[C]\-1\f[] with \f[C]errno\f[] set to the |
|||
respective value. |
|||
Useful for debugging other applications\[aq] behavior to errors. |
|||
T} |
|||
.TE |
|||
.SS Defaults |
|||
.PP |
|||
.TS |
|||
tab(@); |
|||
l l. |
|||
T{ |
|||
Category |
|||
T}@T{ |
|||
Policy |
|||
T} |
|||
_ |
|||
T{ |
|||
action |
|||
T}@T{ |
|||
all |
|||
T} |
|||
T{ |
|||
create |
|||
T}@T{ |
|||
epmfs |
|||
T} |
|||
T{ |
|||
search |
|||
T}@T{ |
|||
ff |
|||
T} |
|||
.TE |
|||
.SS rename |
|||
.PP |
|||
rename (http://man7.org/linux/man-pages/man2/rename.2.html) is a tricky |
|||
function in a merged system. |
|||
Normally if a rename can\[aq]t be done atomically due to the from and to |
|||
paths existing on different mount points it will return \f[C]\-1\f[] |
|||
with \f[C]errno\ =\ EXDEV\f[]. |
|||
The atomic rename is most critical for replacing files in place |
|||
atomically (such as securing writing to a temp file and then replacing a |
|||
target). |
|||
The problem is that by merging multiple paths you can have N instances |
|||
of the source and destinations on different drives. |
|||
Meaning that if you just renamed each source locally you could end up |
|||
with the destination files not overwriten / replaced. |
|||
To address this mergerfs works in the following way. |
|||
If the source and destination exist in different directories it will |
|||
immediately return \f[C]EXDEV\f[]. |
|||
Generally it\[aq]s not expected for cross directory renames to work so |
|||
it should be fine for most instances (mv,rsync,etc.). |
|||
If they do belong to the same directory it then runs the \f[C]rename\f[] |
|||
policy to get the files to rename. |
|||
It iterates through and renames each file while keeping track of those |
|||
paths which have not been renamed. |
|||
If all the renames succeed it will then \f[C]unlink\f[] or |
|||
\f[C]rmdir\f[] the other paths to clean up any preexisting target files. |
|||
This allows the new file to be found without the file itself ever |
|||
disappearing. |
|||
There may still be some issues with this behavior. |
|||
Particularly on error. |
|||
At the moment however this seems the best policy. |
|||
.SS readdir |
|||
.PP |
|||
readdir (http://linux.die.net/man/3/readdir) is very different from most |
|||
functions in this realm. |
|||
It certainly could have it\[aq]s own set of policies to tweak its |
|||
behavior. |
|||
At this time it provides a simple \f[B]first found\f[] merging of |
|||
directories and file found. |
|||
That is: only the first file or directory found for a directory is |
|||
returned. |
|||
Given how FUSE works though the data representing the returned entry |
|||
comes from \f[B]getattr\f[]. |
|||
.PP |
|||
It could be extended to offer the ability to see all files found. |
|||
Perhaps concatenating \f[B]#\f[] and a number to the name. |
|||
But to really be useful you\[aq]d need to be able to access them which |
|||
would complicate file lookup. |
|||
.SS statvfs |
|||
.PP |
|||
statvfs (http://linux.die.net/man/2/statvfs) normalizes the source |
|||
drives based on the fragment size and sums the number of adjusted blocks |
|||
and inodes. |
|||
This means you will see the combined space of all sources. |
|||
Total, used, and free. |
|||
The sources however are dedupped based on the drive so multiple points |
|||
on the same drive will not result in double counting it\[aq]s space. |
|||
.PP |
|||
\f[B]NOTE:\f[] Since we can not (easily) replicate the atomicity of an |
|||
\f[B]mkdir\f[] or \f[B]mknod\f[] without side effects those calls will |
|||
first do a scan to see if the file exists and then attempts a create. |
|||
This means there is a slight race condition. |
|||
Worse case you\[aq]d end up with the directory or file on more than one |
|||
mount. |
|||
.SH BUILDING |
|||
.PP |
|||
\f[B]NOTE:\f[] Prebuilt packages can be found at: |
|||
https://github.com/trapexit/mergerfs/releases |
|||
.PP |
|||
First get the code from github (http://github.com/trapexit/mergerfs). |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
$\ git\ clone\ https://github.com/trapexit/mergerfs.git |
|||
$\ #\ or |
|||
$\ wget\ https://github.com/trapexit/mergerfs/archive/master.zip |
|||
\f[] |
|||
.fi |
|||
.SS Debian / Ubuntu |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
$\ sudo\ apt\-get\ install\ g++\ pkg\-config\ git\ git\-buildpackage\ pandoc\ debhelper\ libfuse\-dev\ libattr1\-dev |
|||
$\ cd\ mergerfs |
|||
$\ make\ deb |
|||
$\ sudo\ dpkg\ \-i\ ../mergerfs_version_arch.deb |
|||
\f[] |
|||
.fi |
|||
.SS Fedora |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
$\ su\ \- |
|||
#\ dnf\ install\ rpm\-build\ fuse\-devel\ libattr\-devel\ pandoc\ gcc\-c++\ git\ make\ which |
|||
#\ cd\ mergerfs |
|||
#\ make\ rpm |
|||
#\ rpm\ \-i\ rpmbuild/RPMS/<arch>/mergerfs\-<verion>.<arch>.rpm |
|||
\f[] |
|||
.fi |
|||
.SS Generically |
|||
.PP |
|||
Have pkg\-config, pandoc, libfuse, libattr1 installed. |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
$\ cd\ mergerfs |
|||
$\ make |
|||
$\ make\ man |
|||
$\ sudo\ make\ install |
|||
\f[] |
|||
.fi |
|||
.SH RUNTIME |
|||
.SS \&.mergerfs pseudo file |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
<mountpoint>/.mergerfs |
|||
\f[] |
|||
.fi |
|||
.PP |
|||
There is a pseudo file available at the mount point which allows for the |
|||
runtime modification of certain \f[B]mergerfs\f[] options. |
|||
The file will not show up in \f[B]readdir\f[] but can be |
|||
\f[B]stat\f[]\[aq]ed and manipulated via |
|||
{list,get,set}xattrs (http://linux.die.net/man/2/listxattr) calls. |
|||
.PP |
|||
Even if xattrs are disabled the |
|||
{list,get,set}xattrs (http://linux.die.net/man/2/listxattr) calls will |
|||
still work. |
|||
.SS Keys |
|||
.PP |
|||
Use \f[C]xattr\ \-l\ /mount/point/.mergerfs\f[] to see all supported |
|||
keys. |
|||
.SS Example |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-l\ .mergerfs |
|||
user.mergerfs.srcmounts:\ /tmp/a:/tmp/b |
|||
user.mergerfs.minfreespace:\ 4294967295 |
|||
user.mergerfs.moveonenospc:\ false |
|||
user.mergerfs.policies:\ all,einval,enosys,enotsup,epmfs,erofs,exdev,ff,ffwp,fwfs,lfs,mfs,newest,rand |
|||
user.mergerfs.version:\ x.y.z |
|||
user.mergerfs.category.action:\ all |
|||
user.mergerfs.category.create:\ epmfs |
|||
user.mergerfs.category.search:\ ff |
|||
user.mergerfs.func.access:\ ff |
|||
user.mergerfs.func.chmod:\ all |
|||
user.mergerfs.func.chown:\ all |
|||
user.mergerfs.func.create:\ epmfs |
|||
user.mergerfs.func.getattr:\ ff |
|||
user.mergerfs.func.getxattr:\ ff |
|||
user.mergerfs.func.link:\ all |
|||
user.mergerfs.func.listxattr:\ ff |
|||
user.mergerfs.func.mkdir:\ epmfs |
|||
user.mergerfs.func.mknod:\ epmfs |
|||
user.mergerfs.func.open:\ ff |
|||
user.mergerfs.func.readlink:\ ff |
|||
user.mergerfs.func.removexattr:\ all |
|||
user.mergerfs.func.rename:\ all |
|||
user.mergerfs.func.rmdir:\ all |
|||
user.mergerfs.func.setxattr:\ all |
|||
user.mergerfs.func.symlink:\ epmfs |
|||
user.mergerfs.func.truncate:\ all |
|||
user.mergerfs.func.unlink:\ all |
|||
user.mergerfs.func.utimens:\ all |
|||
|
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.category.search\ .mergerfs |
|||
ff |
|||
|
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-w\ user.mergerfs.category.search\ ffwp\ .mergerfs |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.category.search\ .mergerfs |
|||
ffwp |
|||
|
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-w\ user.mergerfs.srcmounts\ +/tmp/c\ .mergerfs |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.srcmounts\ .mergerfs |
|||
/tmp/a:/tmp/b:/tmp/c |
|||
|
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-w\ user.mergerfs.srcmounts\ =/tmp/c\ .mergerfs |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.srcmounts\ .mergerfs |
|||
/tmp/c |
|||
|
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-w\ user.mergerfs.srcmounts\ \[aq]+</tmp/a:/tmp/b\[aq]\ .mergerfs |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.srcmounts\ .mergerfs |
|||
/tmp/a:/tmp/b:/tmp/c |
|||
\f[] |
|||
.fi |
|||
.SS user.mergerfs.srcmounts |
|||
.PP |
|||
For \f[B]user.mergerfs.srcmounts\f[] there are several instructions |
|||
available for manipulating the list. |
|||
The value provided is just as the value used at mount time. |
|||
A colon (\[aq]:\[aq]) delimited list of full path globs. |
|||
.PP |
|||
.TS |
|||
tab(@); |
|||
l l. |
|||
T{ |
|||
Instruction |
|||
T}@T{ |
|||
Description |
|||
T} |
|||
_ |
|||
T{ |
|||
[list] |
|||
T}@T{ |
|||
set |
|||
T} |
|||
T{ |
|||
+<[list] |
|||
T}@T{ |
|||
prepend |
|||
T} |
|||
T{ |
|||
+>[list] |
|||
T}@T{ |
|||
append |
|||
T} |
|||
T{ |
|||
\-[list] |
|||
T}@T{ |
|||
remove all values provided |
|||
T} |
|||
T{ |
|||
\-< |
|||
T}@T{ |
|||
remove first in list |
|||
T} |
|||
T{ |
|||
\-> |
|||
T}@T{ |
|||
remove last in list |
|||
T} |
|||
.TE |
|||
.SS minfreespace |
|||
.PP |
|||
Input: interger with an optional suffix. |
|||
\f[B]K\f[], \f[B]M\f[], or \f[B]G\f[]. |
|||
Output: value in bytes |
|||
.SS moveonenospc |
|||
.PP |
|||
Input: \f[B]true\f[] and \f[B]false\f[] Ouput: \f[B]true\f[] or |
|||
\f[B]false\f[] |
|||
.SS categories / funcs |
|||
.PP |
|||
Input: short policy string as described elsewhere in this document |
|||
Output: the policy string except for categories where its funcs have |
|||
multiple types. |
|||
In that case it will be a comma separated list. |
|||
.SS mergerfs file xattrs |
|||
.PP |
|||
While they won\[aq]t show up when using |
|||
listxattr (http://linux.die.net/man/2/listxattr) \f[B]mergerfs\f[] |
|||
offers a number of special xattrs to query information about the files |
|||
served. |
|||
To access the values you will need to issue a |
|||
getxattr (http://linux.die.net/man/2/getxattr) for one of the following: |
|||
.IP \[bu] 2 |
|||
\f[B]user.mergerfs.basepath:\f[] the base mount point for the file given |
|||
the current search policy |
|||
.IP \[bu] 2 |
|||
\f[B]user.mergerfs.relpath:\f[] the relative path of the file from the |
|||
perspective of the mount point |
|||
.IP \[bu] 2 |
|||
\f[B]user.mergerfs.fullpath:\f[] the full path of the original file |
|||
given the search policy |
|||
.IP \[bu] 2 |
|||
\f[B]user.mergerfs.allpaths:\f[] a NUL (\[aq]\[aq]) separated list of |
|||
full paths to all files found |
|||
.IP |
|||
.nf |
|||
\f[C] |
|||
[trapexit:/tmp/mount]\ $\ ls |
|||
A\ B\ C |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.fullpath\ A |
|||
/mnt/a/full/path/to/A |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.basepath\ A |
|||
/mnt/a |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.relpath\ A |
|||
/full/path/to/A |
|||
[trapexit:/tmp/mount]\ $\ xattr\ \-p\ user.mergerfs.allpaths\ A\ |\ tr\ \[aq]\\0\[aq]\ \[aq]\\n\[aq] |
|||
/mnt/a/full/path/to/A |
|||
/mnt/b/full/path/to/A |
|||
\f[] |
|||
.fi |
|||
.SH TOOLING |
|||
.IP \[bu] 2 |
|||
/usr/sbin/fsck.mergerfs: Provides permissions and ownership auditing and |
|||
the ability to fix them. |
|||
.SH TIPS / NOTES |
|||
.IP \[bu] 2 |
|||
If you don\[aq]t see some directories / files you expect in a merged |
|||
point be sure the user has permission to all the underlying directories. |
|||
If \f[C]/drive0/a\f[] has is owned by \f[C]root:root\f[] with ACLs set |
|||
to \f[C]0700\f[] and \f[C]/drive1/a\f[] is \f[C]root:root\f[] and |
|||
\f[C]0755\f[] you\[aq]ll see only \f[C]/drive1/a\f[]. |
|||
Use \f[C]fsck.mergerfs\f[] to audit the drive for out of sync |
|||
permissions. |
|||
.IP \[bu] 2 |
|||
Since POSIX gives you only error or success on calls its difficult to |
|||
determine the proper behavior when applying the behavior to multiple |
|||
targets. |
|||
Generally if something succeeds when reading it returns the data it can. |
|||
If something fails when making an action we continue on and return the |
|||
last error. |
|||
.IP \[bu] 2 |
|||
The recommended options are \f[B]defaults,allow_other\f[]. |
|||
The \f[B]allow_other\f[] is to allow users who are not the one which |
|||
executed mergerfs access to the mountpoint. |
|||
\f[B]defaults\f[] is described above and should offer the best |
|||
performance. |
|||
It\[aq]s possible that if you\[aq]re running on an older platform the |
|||
\f[B]splice\f[] features aren\[aq]t available and could error. |
|||
In that case simply use the other options manually. |
|||
.IP \[bu] 2 |
|||
If write performance is valued more than read it may be useful to enable |
|||
\f[B]direct_io\f[]. |
|||
.IP \[bu] 2 |
|||
Remember that some policies mixed with some functions may result in |
|||
strange behaviors. |
|||
Not that some of these behaviors and race conditions couldn\[aq]t happen |
|||
outside \f[B]mergerfs\f[] but that they are far more likely to occur on |
|||
account of attempt to merge together multiple sources of data which |
|||
could be out of sync due to the different policies. |
|||
.IP \[bu] 2 |
|||
An example: Kodi (http://kodi.tv) and Plex (http://plex.tv) can |
|||
apparently use directory mtime (http://linux.die.net/man/2/stat) to more |
|||
efficiently determine whether or not to scan for new content rather than |
|||
simply performing a full scan. |
|||
If using the current default \f[B]getattr\f[] policy of \f[B]ff\f[] its |
|||
possible \f[B]Kodi\f[] will miss an update on account of it returning |
|||
the first directory found\[aq]s \f[B]stat\f[] info and its a later |
|||
directory on another mount which had the \f[B]mtime\f[] recently |
|||
updated. |
|||
To fix this you will want to set \f[B]func.getattr=newest\f[]. |
|||
Remember though that this is just \f[B]stat\f[]. |
|||
If the file is later \f[B]open\f[]\[aq]ed or \f[B]unlink\f[]\[aq]ed and |
|||
the policy is different for those then a completely different file or |
|||
directory could be acted on. |
|||
.IP \[bu] 2 |
|||
Due to previously mentioned issues its generally best to set |
|||
\f[B]category\f[] wide policies rather than individual |
|||
\f[B]func\f[]\[aq]s. |
|||
This will help limit the confusion of tools such as |
|||
rsync (http://linux.die.net/man/1/rsync). |
|||
.SH Known Issues / Bugs |
|||
.SS Samba |
|||
.IP \[bu] 2 |
|||
Moving files or directories between directories on a SMB share fail with |
|||
IO errors. |
|||
.RS 2 |
|||
.PP |
|||
Workaround: Copy the file/directory and then remove the original rather |
|||
than move. |
|||
.PP |
|||
This isn\[aq]t an issue with Samba but some SMB clients. |
|||
GVFS\-fuse v1.20.3 and prior (found in Ubuntu 14.04 among others) failed |
|||
to handle certain error codes correctly. |
|||
Particularly \f[B]STATUS_NOT_SAME_DEVICE\f[] which comes from the |
|||
\f[B]EXDEV\f[] which is returned by \f[B]rename\f[] when the call is |
|||
crossing mountpoints. |
|||
When a program gets an \f[B]EXDEV\f[] it needs to explicitly take an |
|||
alternate action to accomplish it\[aq]s goal. |
|||
In the case of \f[B]mv\f[] or similar it tries \f[B]rename\f[] and on |
|||
\f[B]EXDEV\f[] falls back to a manual copying of data between the two |
|||
locations and unlinking the source. |
|||
In these older versions of GVFS\-fuse if it received \f[B]EXDEV\f[] it |
|||
would translate that into \f[B]EIO\f[]. |
|||
This would cause \f[B]mv\f[] or most any application attempting to move |
|||
files around on that SMB share to fail with a IO error. |
|||
.PP |
|||
GVFS\-fuse v1.22.0 (https://bugzilla.gnome.org/show_bug.cgi?id=734568) |
|||
and above fixed this issue but a large number of systems use the older |
|||
release. |
|||
On Ubuntu the version can be checked by issuing |
|||
\f[C]apt\-cache\ showpkg\ gvfs\-fuse\f[]. |
|||
Most distros released in 2015 seem to have the updated release and will |
|||
work fine but older systems may not. |
|||
Upgrading gvfs\-fuse or the distro in general will address the problem. |
|||
.PP |
|||
In Apple\[aq]s MacOSX 10.9 they replaced Samba (client and server) with |
|||
their own product. |
|||
It appears their new client does not handle \f[B]EXDEV\f[] either and |
|||
responds similar to older release of gvfs on Linux. |
|||
.RE |
|||
.SS Supplemental groups |
|||
.IP \[bu] 2 |
|||
Due to the overhead of |
|||
getgroups/setgroups (http://linux.die.net/man/2/setgroups) mergerfs |
|||
utilizes a cache. |
|||
This cache is opportunistic and per thread. |
|||
Each thread will query the supplemental groups for a user when that |
|||
particular thread needs to change credentials and will keep that data |
|||
for the lifetime of the mount or thread. |
|||
This means that if a user is added to a group it may not be picked up |
|||
without the restart of mergerfs. |
|||
However, since the high level FUSE API\[aq]s (at least the standard |
|||
version) thread pool dynamically grows and shrinks it\[aq]s possible |
|||
that over time a thread will be killed and later a new thread with no |
|||
cache will start and query the new data. |
|||
.RS 2 |
|||
.PP |
|||
The gid cache uses fixed storage to simplify the design and be |
|||
compatible with older systems which may not have C++11 compilers (as the |
|||
original design required). |
|||
There is enough storage for 256 users\[aq] supplemental groups. |
|||
Each user is allowed upto 32 supplemental groups. |
|||
Linux >= 2.6.3 allows upto 65535 groups per user but most other *nixs |
|||
allow far less. |
|||
NFS allowing only 16. |
|||
The system does handle overflow gracefully. |
|||
If the user has more than 32 supplemental groups only the first 32 will |
|||
be used. |
|||
If more than 256 users are using the system when an uncached user is |
|||
found it will evict an existing user\[aq]s cache at random. |
|||
So long as there aren\[aq]t more than 256 active users this should be |
|||
fine. |
|||
If either value is too low for your needs you will have to modify |
|||
\f[C]gidcache.hpp\f[] to increase the values. |
|||
Note that doing so will increase the memory needed by each thread. |
|||
.RE |
|||
.SH FAQ |
|||
.PP |
|||
\f[I]It\[aq]s mentioned that there are some security issues with mhddfs. |
|||
What are they? How does mergerfs address them?\f[] |
|||
.PP |
|||
mhddfs (https://github.com/trapexit/mhddfs) tries to handle being run as |
|||
\f[B]root\f[] by calling |
|||
getuid() (https://github.com/trapexit/mhddfs/blob/cae96e6251dd91e2bdc24800b4a18a74044f6672/src/main.c#L319) |
|||
and if it returns \f[B]0\f[] then it will |
|||
chown (http://linux.die.net/man/1/chown) the file. |
|||
Not only is that a race condition but it doesn\[aq]t handle many other |
|||
situations. |
|||
Rather than attempting to simulate POSIX ACL behaviors the proper |
|||
behavior is to use seteuid (http://linux.die.net/man/2/seteuid) and |
|||
setegid (http://linux.die.net/man/2/setegid), become the user making the |
|||
original call and perform the action as them. |
|||
This is how mergerfs (https://github.com/trapexit/mergerfs) handles |
|||
things. |
|||
.PP |
|||
If you are familiar with POSIX standards you\[aq]ll know that this |
|||
behavior poses a problem. |
|||
\f[B]seteuid\f[] and \f[B]setegid\f[] affect the whole process and |
|||
\f[B]libfuse\f[] is multithreaded by default. |
|||
We\[aq]d need to lock access to \f[B]seteuid\f[] and \f[B]setegid\f[] |
|||
with a mutex so that the several threads aren\[aq]t stepping on one |
|||
another and files end up with weird permissions and ownership. |
|||
This however wouldn\[aq]t scale well. |
|||
With lots of calls the contention on that mutex would be extremely high. |
|||
Thankfully on Linux and OSX we have a better solution. |
|||
.PP |
|||
OSX has a non\-portable pthread |
|||
extension (https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/pthread_setugid_np.2.html) |
|||
for per\-thread user and group impersonation. |
|||
.PP |
|||
Linux does not support |
|||
pthread_setugid_np (https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/pthread_setugid_np.2.html) |
|||
but user and group IDs are a per\-thread attribute though documentation |
|||
on that fact or how to manipulate them is not well distributed. |
|||
From the \f[B]4.00\f[] release of the Linux man\-pages project for |
|||
setuid (http://man7.org/linux/man-pages/man2/setuid.2.html) |
|||
.RS |
|||
.PP |
|||
At the kernel level, user IDs and group IDs are a per\-thread attribute. |
|||
However, POSIX requires that all threads in a process share the same |
|||
credentials. |
|||
The NPTL threading implementation handles the POSIX requirements by |
|||
providing wrapper functions for the various system calls that change |
|||
process UIDs and GIDs. |
|||
These wrapper functions (including the one for setuid()) employ a |
|||
signal\-based technique to ensure that when one thread changes |
|||
credentials, all of the other threads in the process also change their |
|||
credentials. |
|||
For details, see nptl(7). |
|||
.RE |
|||
.PP |
|||
Turns out the setreuid syscalls apply only to the thread. |
|||
GLIBC hides this away using RT signals to inform all threads to change |
|||
credentials. |
|||
Taking after \f[B]Samba\f[] mergerfs uses |
|||
\f[B]syscall(SYS_setreuid,...)\f[] to set the callers credentials for |
|||
that thread only. |
|||
Jumping back to \f[B]root\f[] as necessary should escalated privileges |
|||
be needed (for instance: to clone paths). |
|||
.PP |
|||
For non\-Linux systems mergerfs uses a read\-write lock and changes |
|||
credentials only when necessary. |
|||
If multiple threads are to be user X then only the first one will need |
|||
to change the processes credentials. |
|||
So long as the other threads need to be user X they will take a readlock |
|||
allow multiple threads to share the credentials. |
|||
Once a request comes in to run as user Y that thread will attempt a |
|||
write lock and change to Y\[aq]s credentials when it can. |
|||
If the ability to give writers priority is supported then that flag will |
|||
be used so threads trying to change credentials don\[aq]t starve. |
|||
This isn\[aq]t the best solution but should work reasonably well. |
|||
As new platforms are supported if they offer per thread credentials |
|||
those APIs will be adopted. |
|||
.SH AUTHORS |
|||
Antonio SJ Musumeci <trapexit@spawn.link>. |
Write
Preview
Loading…
Cancel
Save
Reference in new issue