Mirror
/
mergerfs
mirror of https://github.com/trapexit/mergerfs.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

Merge pull request #574 from trapexit/opts 

remove `defaults`, hard code atomic_o_trunc, big_writes, and default_…
pull/575/head
trapexit 6 years ago
committed by GitHub
parent
a0bb2f8480 c5b2415daf
commit
5a1a60742e
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 35 additions and 58 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 23
      README.md
  2. 44
      man/mergerfs.1
  3. 5
      src/init.cpp
  4. 4
      src/ioctl.cpp
  5. 17
      src/option_parser.cpp

23
README.md
View File

@ -63,12 +63,11 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs**
### mount options ### mount options
* **defaults**: a shortcut for FUSE's **atomic_o_trunc**, **auto_cache**, **big_writes**, **default_permissions**, **splice_move**, **splice_read**, and **splice_write**. These options seem to provide the best performance.
* **allow_other**: a libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases. * **allow_other**: a libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases.
* **direct_io**: causes FUSE to bypass caching which can increase write speeds at the detriment of reads. Note that not enabling `direct_io` will cause double caching of files and therefore less memory for caching generally (enable **dropcacheonclose** to help with this problem). However, `mmap` does not work when `direct_io` is enabled. * **direct_io**: causes FUSE to bypass caching which can increase write speeds at the detriment of reads. Note that not enabling `direct_io` will cause double caching of files and therefore less memory for caching generally (enable **dropcacheonclose** to help with this problem). However, `mmap` does not work when `direct_io` is enabled.
* **minfreespace=value**: the minimum space value used for creation policies. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G) * **minfreespace=value**: the minimum space value used for creation policies. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G)
* **moveonenospc=true|false**: when enabled (set to **true**) if a **write** fails with **ENOSPC** or **EDQUOT** a scan of all drives will be done looking for the drive with the most free space which is at least the size of the file plus the amount which failed to write. An attempt to move the file to that drive will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false) * **moveonenospc=true|false**: when enabled (set to **true**) if a **write** fails with **ENOSPC** or **EDQUOT** a scan of all drives will be done looking for the drive with the most free space which is at least the size of the file plus the amount which failed to write. An attempt to move the file to that drive will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false)
* **use_ino**: causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is generally recommended it be enabled so that hard linked files share the same inode value.
* **use_ino**: causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.
* **hard_remove**: force libfuse to immedately remove files when unlinked. This will keep the `.fuse_hidden` files from showing up but if software uses an opened but unlinked file in certain ways it could result in errors. * **hard_remove**: force libfuse to immedately remove files when unlinked. This will keep the `.fuse_hidden` files from showing up but if software uses an opened but unlinked file in certain ways it could result in errors.
* **dropcacheonclose=true|false**: when a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **direct_io** is not enabled to limit double caching. (default: false) * **dropcacheonclose=true|false**: when a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **direct_io** is not enabled to limit double caching. (default: false)
* **symlinkify=true|false**: when enabled (set to **true**) and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false) * **symlinkify=true|false**: when enabled (set to **true**) and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false)
@ -99,7 +98,7 @@ To make it easier to include multiple branches mergerfs supports [globbing](http
Each branch can have a suffix of `=RW` (read / write), `=RO` (read-only), or `=NC` (no create). These suffixes work with globs as well and will apply to each path found. `RW` is the default behavior and those paths will be eligible for all policy categories. `RO` will exclude those paths from `create` and `action` policies (just as a filesystem being mounted `ro` would). `NC` will exclude those paths from `create` policies (you can't create but you can change / delete). Each branch can have a suffix of `=RW` (read / write), `=RO` (read-only), or `=NC` (no create). These suffixes work with globs as well and will apply to each path found. `RW` is the default behavior and those paths will be eligible for all policy categories. `RO` will exclude those paths from `create` and `action` policies (just as a filesystem being mounted `ro` would). `NC` will exclude those paths from `create` policies (you can't create but you can change / delete).
``` ```
$ mergerfs -o defaults,allow_other,use_ino /mnt/disk\*:/mnt/cdrom /media/drives
# mergerfs -o allow_other,use_ino /mnt/disk\*:/mnt/cdrom /media/drives
``` ```
The above line will use all mount points in /mnt prefixed with **disk** and the **cdrom**. The above line will use all mount points in /mnt prefixed with **disk** and the **cdrom**.
@ -107,8 +106,8 @@ The above line will use all mount points in /mnt prefixed with **disk** and the
To have the pool mounted at boot or otherwise accessable from related tools use **/etc/fstab**. To have the pool mounted at boot or otherwise accessable from related tools use **/etc/fstab**.
``` ```
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/disk*:/mnt/cdrom /media/drives fuse.mergerfs defaults,allow_other,use_ino 0 0
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/disk*:/mnt/cdrom /media/drives fuse.mergerfs allow_other,use_ino 0 0
``` ```
**NOTE:** the globbing is done at mount or xattr update time (see below). If a new directory is added matching the glob after the fact it will not be automatically included. **NOTE:** the globbing is done at mount or xattr update time (see below). If a new directory is added matching the glob after the fact it will not be automatically included.
@ -588,8 +587,8 @@ done
# TIPS / NOTES # TIPS / NOTES
* The recommended base options are **defaults,allow_other,direct_io,use_ino**. (**use_ino** will only work when used with mergerfs 2.18.0 and above.)
* Run mergerfs as `root` unless you're merging paths which are owned by the same user otherwise strange permission issues may arise.
* **use_ino** will only work when used with mergerfs 2.18.0 and above.
* Run mergerfs as `root` (with **allow_other**) unless you're merging paths which are owned by the same user otherwise strange permission issues may arise.
* https://github.com/trapexit/backup-and-recovery-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure. * https://github.com/trapexit/backup-and-recovery-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure.
* If you don't see some directories and files you expect in a merged point or policies seem to skip drives be sure the user has permission to all the underlying directories. Use `mergerfs.fsck` to audit the drive for out of sync permissions. * If you don't see some directories and files you expect in a merged point or policies seem to skip drives be sure the user has permission to all the underlying directories. Use `mergerfs.fsck` to audit the drive for out of sync permissions.
* Do **not** use `direct_io` if you expect applications (such as rtorrent) to [mmap](http://linux.die.net/man/2/mmap) files. It is not currently supported in FUSE w/ `direct_io` enabled. Enabling `dropcacheonclose` is recommended when `direct_io` is disabled. * Do **not** use `direct_io` if you expect applications (such as rtorrent) to [mmap](http://linux.die.net/man/2/mmap) files. It is not currently supported in FUSE w/ `direct_io` enabled. Enabling `dropcacheonclose` is recommended when `direct_io` is disabled.
@ -758,7 +757,7 @@ https://lkml.org/lkml/2016/9/14/527
[25192.603193] [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0 [25192.603193] [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0
``` ```
There is a bug in the kernel. A work around appears to be turning off `splice`. Add `no_splice_write,no_splice_move,no_splice_read` to mergerfs' options. Should be placed after `defaults` if it is used since it will turn them on. This however is not guaranteed to work.
There is a bug in the kernel. A work around appears to be turning off `splice`. Don't add the `splice_*` arguments or add `no_splice_write,no_splice_move,no_splice_read`. This, however, is not guaranteed to work.
#### rm: fts_read failed: No such file or directory #### rm: fts_read failed: No such file or directory
@ -830,7 +829,7 @@ Below is an example of mhddfs and mergerfs setup to work similarly.
`mhddfs -o mlimit=4G,allow_other /mnt/drive1,/mnt/drive2 /mnt/pool` `mhddfs -o mlimit=4G,allow_other /mnt/drive1,/mnt/drive2 /mnt/pool`
`mergerfs -o minfreespace=4G,defaults,allow_other,category.create=ff /mnt/drive1:/mnt/drive2 /mnt/pool`
`mergerfs -o minfreespace=4G,allow_other,category.create=ff /mnt/drive1:/mnt/drive2 /mnt/pool`
#### Why use mergerfs over aufs? #### Why use mergerfs over aufs?
@ -935,9 +934,9 @@ For non-Linux systems mergerfs uses a read-write lock and changes credentials on
# PERFORMANCE TWEAKING # PERFORMANCE TWEAKING
* try adding (or removing) `direct_io` * try adding (or removing) `direct_io`
* try adding (or removing) `auto_cache` / `noauto_cache` (included in `defaults`)
* try adding (or removing) `kernel_cache` (don't use the underlying filesystems directly if enabling `kernel_cache`)
* try adding (or removing) `splice_move`, `splice_read`, and `splice_write` (all three included in `defaults`)
* try adding (or removing) `auto_cache`
* try adding (or removing) `kernel_cache`
* try adding (or removing) `splice_move`, `splice_read`, and `splice_write`
* try increasing cache timeouts `attr_timeout`, `entry_timeout`, `ac_attr_timeout`, `negative_timeout` * try increasing cache timeouts `attr_timeout`, `entry_timeout`, `ac_attr_timeout`, `negative_timeout`
* try changing the number of worker threads * try changing the number of worker threads
* try disabling `security_capability` or `xattr` * try disabling `security_capability` or `xattr`

44
man/mergerfs.1
View File

@ -78,11 +78,6 @@ so you can mix rw and ro drives.
.SH OPTIONS .SH OPTIONS
.SS mount options .SS mount options
.IP \[bu] 2 .IP \[bu] 2
\f[B]defaults\f[]: a shortcut for FUSE\[aq]s \f[B]atomic_o_trunc\f[],
\f[B]auto_cache\f[], \f[B]big_writes\f[], \f[B]default_permissions\f[],
\f[B]splice_move\f[], \f[B]splice_read\f[], and \f[B]splice_write\f[].
These options seem to provide the best performance.
.IP \[bu] 2
\f[B]allow_other\f[]: a libfuse option which allows users besides the \f[B]allow_other\f[]: a libfuse option which allows users besides the
one which ran mergerfs to see the filesystem. one which ran mergerfs to see the filesystem.
This is required for most use\-cases. This is required for most use\-cases.
@ -112,8 +107,8 @@ write retried.
.IP \[bu] 2 .IP \[bu] 2
\f[B]use_ino\f[]: causes mergerfs to supply file/directory inodes rather \f[B]use_ino\f[]: causes mergerfs to supply file/directory inodes rather
than libfuse. than libfuse.
While not a default it is generally recommended it be enabled so that
hard linked files share the same inode value.
While not a default it is recommended it be enabled so that linked files
share the same inode value.
.IP \[bu] 2 .IP \[bu] 2
\f[B]hard_remove\f[]: force libfuse to immedately remove files when \f[B]hard_remove\f[]: force libfuse to immedately remove files when
unlinked. unlinked.
@ -253,7 +248,7 @@ can\[aq]t create but you can change / delete).
.IP .IP
.nf .nf
\f[C] \f[C]
$\ mergerfs\ \-o\ defaults,allow_other,use_ino\ /mnt/disk\\*:/mnt/cdrom\ /media/drives
#\ mergerfs\ \-o\ allow_other,use_ino\ /mnt/disk\\*:/mnt/cdrom\ /media/drives
\f[] \f[]
.fi .fi
.PP .PP
@ -265,8 +260,8 @@ tools use \f[B]/etc/fstab\f[].
.IP .IP
.nf .nf
\f[C] \f[C]
#\ <file\ system>\ \ \ \ \ \ \ \ <mount\ point>\ \ <type>\ \ \ \ \ \ \ \ \ <options>\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ <dump>\ \ <pass>
/mnt/disk*:/mnt/cdrom\ \ /media/drives\ \ fuse.mergerfs\ \ defaults,allow_other,use_ino\ \ \ 0\ \ \ \ \ \ \ 0
#\ <file\ system>\ \ \ \ \ \ \ \ <mount\ point>\ \ <type>\ \ \ \ \ \ \ \ \ <options>\ \ \ \ \ \ \ \ \ \ \ \ \ <dump>\ \ <pass>
/mnt/disk*:/mnt/cdrom\ \ /media/drives\ \ fuse.mergerfs\ \ allow_other,use_ino\ \ \ 0\ \ \ \ \ \ \ 0
\f[] \f[]
.fi .fi
.PP .PP
@ -1204,13 +1199,12 @@ done
.fi .fi
.SH TIPS / NOTES .SH TIPS / NOTES
.IP \[bu] 2 .IP \[bu] 2
The recommended base options are
\f[B]defaults,allow_other,direct_io,use_ino\f[].
(\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and
above.)
\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and
above.
.IP \[bu] 2 .IP \[bu] 2
Run mergerfs as \f[C]root\f[] unless you\[aq]re merging paths which are
owned by the same user otherwise strange permission issues may arise.
Run mergerfs as \f[C]root\f[] (with \f[B]allow_other\f[]) unless
you\[aq]re merging paths which are owned by the same user otherwise
strange permission issues may arise.
.IP \[bu] 2 .IP \[bu] 2
https://github.com/trapexit/backup\-and\-recovery\-howtos : A set of https://github.com/trapexit/backup\-and\-recovery\-howtos : A set of
guides / howtos on creating a data storage system, backing it up, guides / howtos on creating a data storage system, backing it up,
@ -1538,11 +1532,9 @@ https://lkml.org/lkml/2016/9/14/527
.PP .PP
There is a bug in the kernel. There is a bug in the kernel.
A work around appears to be turning off \f[C]splice\f[]. A work around appears to be turning off \f[C]splice\f[].
Add \f[C]no_splice_write,no_splice_move,no_splice_read\f[] to
mergerfs\[aq] options.
Should be placed after \f[C]defaults\f[] if it is used since it will
turn them on.
This however is not guaranteed to work.
Don\[aq]t add the \f[C]splice_*\f[] arguments or add
\f[C]no_splice_write,no_splice_move,no_splice_read\f[].
This, however, is not guaranteed to work.
.SS rm: fts_read failed: No such file or directory .SS rm: fts_read failed: No such file or directory
.PP .PP
Not \f[I]really\f[] a bug. Not \f[I]really\f[] a bug.
@ -1671,7 +1663,7 @@ Below is an example of mhddfs and mergerfs setup to work similarly.
.PP .PP
\f[C]mhddfs\ \-o\ mlimit=4G,allow_other\ /mnt/drive1,/mnt/drive2\ /mnt/pool\f[] \f[C]mhddfs\ \-o\ mlimit=4G,allow_other\ /mnt/drive1,/mnt/drive2\ /mnt/pool\f[]
.PP .PP
\f[C]mergerfs\ \-o\ minfreespace=4G,defaults,allow_other,category.create=ff\ /mnt/drive1:/mnt/drive2\ /mnt/pool\f[]
\f[C]mergerfs\ \-o\ minfreespace=4G,allow_other,category.create=ff\ /mnt/drive1:/mnt/drive2\ /mnt/pool\f[]
.SS Why use mergerfs over aufs? .SS Why use mergerfs over aufs?
.PP .PP
aufs is mostly abandoned and no longer available in many distros. aufs is mostly abandoned and no longer available in many distros.
@ -1868,14 +1860,12 @@ assuming there are few users.
.IP \[bu] 2 .IP \[bu] 2
try adding (or removing) \f[C]direct_io\f[] try adding (or removing) \f[C]direct_io\f[]
.IP \[bu] 2 .IP \[bu] 2
try adding (or removing) \f[C]auto_cache\f[] / \f[C]noauto_cache\f[]
(included in \f[C]defaults\f[])
try adding (or removing) \f[C]auto_cache\f[]
.IP \[bu] 2 .IP \[bu] 2
try adding (or removing) \f[C]kernel_cache\f[] (don\[aq]t use the
underlying filesystems directly if enabling \f[C]kernel_cache\f[])
try adding (or removing) \f[C]kernel_cache\f[]
.IP \[bu] 2 .IP \[bu] 2
try adding (or removing) \f[C]splice_move\f[], \f[C]splice_read\f[], and try adding (or removing) \f[C]splice_move\f[], \f[C]splice_read\f[], and
\f[C]splice_write\f[] (all three included in \f[C]defaults\f[])
\f[C]splice_write\f[]
.IP \[bu] 2 .IP \[bu] 2
try increasing cache timeouts \f[C]attr_timeout\f[], try increasing cache timeouts \f[C]attr_timeout\f[],
\f[C]entry_timeout\f[], \f[C]ac_attr_timeout\f[], \f[C]entry_timeout\f[], \f[C]ac_attr_timeout\f[],

5
src/init.cpp
View File

@ -28,10 +28,11 @@ namespace mergerfs
{ {
ugid::init(); ugid::init();
conn->want |= FUSE_CAP_ASYNC_READ;
conn->want |= FUSE_CAP_ATOMIC_O_TRUNC;
conn->want |= FUSE_CAP_BIG_WRITES;
conn->want |= FUSE_CAP_DONT_MASK; conn->want |= FUSE_CAP_DONT_MASK;
#ifdef FUSE_CAP_IOCTL_DIR
conn->want |= FUSE_CAP_IOCTL_DIR; conn->want |= FUSE_CAP_IOCTL_DIR;
#endif
return &Config::get_writable(); return &Config::get_writable();
} }

4
src/ioctl.cpp
View File

@ -62,7 +62,6 @@ namespace local
return local::ioctl(fi->fd,cmd_,data_); return local::ioctl(fi->fd,cmd_,data_);
} }
#ifdef FUSE_IOCTL_DIR
#ifndef O_NOATIME #ifndef O_NOATIME
#define O_NOATIME 0 #define O_NOATIME 0
@ -119,7 +118,6 @@ namespace local
cmd_, cmd_,
data_); data_);
} }
#endif
} }
namespace mergerfs namespace mergerfs
@ -134,10 +132,8 @@ namespace mergerfs
unsigned int flags_, unsigned int flags_,
void *data_) void *data_)
{ {
#ifdef FUSE_IOCTL_DIR
if(flags_ & FUSE_IOCTL_DIR) if(flags_ & FUSE_IOCTL_DIR)
return local::ioctl_dir(ffi_,cmd_,data_); return local::ioctl_dir(ffi_,cmd_,data_);
#endif
return local::ioctl_file(ffi_,cmd_,data_); return local::ioctl_file(ffi_,cmd_,data_);
} }

17
src/option_parser.cpp
View File

@ -52,11 +52,9 @@ void
set_option(fuse_args &args, set_option(fuse_args &args,
const std::string &option_) const std::string &option_)
{ {
string option;
option = "-o" + option_;
fuse_opt_insert_arg(&args,1,option.c_str());
fuse_opt_add_arg(&args,"-o");
fuse_opt_add_arg(&args,option_.c_str());
} }
static static
@ -103,12 +101,8 @@ void
set_default_options(fuse_args &args) set_default_options(fuse_args &args)
{ {
set_option(args,"atomic_o_trunc"); set_option(args,"atomic_o_trunc");
set_option(args,"auto_cache");
set_option(args,"big_writes"); set_option(args,"big_writes");
set_option(args,"default_permissions"); set_option(args,"default_permissions");
set_option(args,"splice_move");
set_option(args,"splice_read");
set_option(args,"splice_write");
} }
static static
@ -240,7 +234,7 @@ parse_and_process_arg(Config &config,
fuse_args *outargs) fuse_args *outargs)
{ {
if(arg == "defaults") if(arg == "defaults")
return (set_default_options(*outargs),0);
return 0;
else if(arg == "direct_io") else if(arg == "direct_io")
return (config.direct_io=true,0); return (config.direct_io=true,0);
@ -363,10 +357,6 @@ usage(void)
"mergerfs options:\n" "mergerfs options:\n"
" <srcpaths> ':' delimited list of directories. Supports\n" " <srcpaths> ':' delimited list of directories. Supports\n"
" shell globbing (must be escaped in shell)\n" " shell globbing (must be escaped in shell)\n"
" -o defaults Default FUSE options which seem to provide the\n"
" best performance: atomic_o_trunc, auto_cache,\n"
" big_writes, default_permissions, splice_read,\n"
" splice_write, splice_move\n"
" -o func.<f>=<p> Set function <f> to policy <p>\n" " -o func.<f>=<p> Set function <f> to policy <p>\n"
" -o category.<c>=<p> Set functions in category <c> to <p>\n" " -o category.<c>=<p> Set functions in category <c> to <p>\n"
" -o cache.open=<int> 'open' policy cache timeout in seconds.\n" " -o cache.open=<int> 'open' policy cache timeout in seconds.\n"
@ -488,6 +478,7 @@ namespace mergerfs
opts, opts,
::option_processor); ::option_processor);
set_default_options(args);
set_fsname(args,config.branches); set_fsname(args,config.branches);
set_subtype(args); set_subtype(args);
} }

Write Preview
Loading…
Cancel
Save