From e9e17baf5e1282a6a2b2358de298461b1cb7f797 Mon Sep 17 00:00:00 2001 From: Antonio SJ Musumeci Date: Tue, 29 Dec 2020 19:34:13 -0500 Subject: [PATCH 1/6] README: misc updates and tweaks --- README.md | 116 +++++++++++++------------- man/mergerfs.1 | 219 +++++++++++++++++++++++++------------------------ 2 files changed, 173 insertions(+), 162 deletions(-) diff --git a/README.md b/README.md index 38873c98..af27bce8 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ mergerfs -o<options> <branches> <mountpoint> * Handles pool of read-only and read/write drives * Can turn read-only files into symlinks to underlying file * Hard link copy-on-write / CoW -* supports POSIX ACLs +* Support for POSIX ACLs # HOW IT WORKS @@ -59,7 +59,7 @@ A + B = C +-- file6 ``` -mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives. It also does **not** split data across drives. It is not RAID0 / striping. It is simply a union. +mergerfs does **NOT** support the copy-on-write (CoW) or whiteout behaviors found in **aufs** and **overlayfs**. You can **not** mount a read-only filesystem and write to it. However, mergerfs will ignore read-only drives when creating new files so you can mix read-write and read-only drives. It also does **NOT** split data across drives. It is not RAID0 / striping. It is simply a union of other filesystems. # TERMINOLOGY @@ -67,9 +67,9 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** * branch: A base path used in the pool. * pool: The mergerfs mount. The union of the branches. * relative path: The path in the pool relative to the branch and mount. +* function: A filesystem call (open, unlink, create, getattr, rmdir, etc.) +* category: A collection of functions based on basic behavior (action, create, search). * policy: The algorithm used to select a file when performing a function. -* function: A filesystem call (open, unlink, create, getattr, etc.) -* category: A collection of functions (action, create, search). * path preservation: Aspect of some policies which includes checking the path for which a file would be created. @@ -77,22 +77,25 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs** If you don't already know that you have a special use case then just start with one of the following option sets. -#### You don't need `mmap` +#### You need `mmap` (used by rtorrent and many sqlite3 base software) + +`allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs` -`use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs` +#### You don't need `mmap` -#### You do need `mmap` (used by rtorrent and some other programs) +`allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs` -`use_ino,cache.files=partial,dropcacheonclose=true,allow_other,category.create=mfs` See the mergerfs [wiki for real world deployments](https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments) for comparisons / ideas. # OPTIONS +These options are the same regardless you use them with the `mergerfs` commandline program, used in fstab, or in a config file. + ### mount options -* **config**: Path to a config file. Same arguments as below in key=val format. +* **config**: Path to a config file. Same arguments as below in key=val / ini style format. * **branches**: Colon delimited list of branches. * **allow_other**: A libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases. * **minfreespace=SIZE**: The minimum space value used for creation policies. Can be overridden by branch specific option. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G) @@ -101,7 +104,7 @@ See the mergerfs [wiki for real world deployments](https://github.com/trapexit/m * **inodecalc=passthrough|path-hash|devino-hash|hybrid-hash**: Selects the inode calculation algorithm. (default: hybrid-hash) * **dropcacheonclose=BOOL**: When a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **cache.files=partial|full|auto-full** to limit double caching. (default: false) * **symlinkify=BOOL**: When enabled and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false) -* **symlinkify_timeout=INT**: Time to wait, in seconds, to activate the **symlinkify** behavior. (default: 3600) +* **symlinkify_timeout=UINT**: Time to wait, in seconds, to activate the **symlinkify** behavior. (default: 3600) * **nullrw=BOOL**: Turns reads and writes into no-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false) * **ignorepponrename=BOOL**: Ignore path preserving on rename. Typically rename and link act differently depending on the policy of `create` (read below). Enabling this will cause rename and link to always use the non-path preserving behavior. This means files, when renamed or linked, will stay on the same drive. (default: false) * **security_capability=BOOL**: If false return ENOATTR when xattr security.capability is queried. (default: true) @@ -112,16 +115,18 @@ See the mergerfs [wiki for real world deployments](https://github.com/trapexit/m * **nfsopenhack=off|git|all**: A workaround for exporting mergerfs over NFS where there are issues with creating files for write while setting the mode to read-only. (default: off) * **posix_acl=BOOL**: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false) * **async_read=BOOL**: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true) -* **fuse_msg_size=INT**: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256) +* **fuse_msg_size=UINT**: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256) * **threads=INT**: Number of threads to use in multithreaded mode. When set to zero it will attempt to discover and use the number of logical cores. If the lookup fails it will fall back to using 4. If the thread count is set negative it will look up the number of cores then divide by the absolute value. ie. threads=-2 on an 8 core machine will result in 8 / 2 = 4 threads. There will always be at least 1 thread. NOTE: higher number of threads increases parallelism but usually decreases throughput. (default: 0) * **fsname=STR**: Sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed. * **func.FUNC=POLICY**: Sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest** -* **category.CATEGORY=POLICY**: Sets policy of all FUSE functions in the provided category. See POLICIES section for defaults. Example: **category.create=mfs** -* **cache.open=INT**: 'open' policy cache timeout in seconds. (default: 0) -* **cache.statfs=INT**: 'statfs' cache timeout in seconds. (default: 0) -* **cache.attr=INT**: File attribute cache timeout in seconds. (default: 1) -* **cache.entry=INT**: File name lookup cache timeout in seconds. (default: 1) -* **cache.negative_entry=INT**: Negative file name lookup cache timeout in seconds. (default: 0) +* **category.action=POLICY**: Sets policy of all FUSE functions in the action category. (default: epall) +* **category.create=POLICY**: Sets policy of all FUSE functions in the create category. (default: epmfs) +* **category.search=POLICY**: Sets policy of all FUSE functions in the search category. (default: ff) +* **cache.open=UINT**: 'open' policy cache timeout in seconds. (default: 0) +* **cache.statfs=UINT**: 'statfs' cache timeout in seconds. (default: 0) +* **cache.attr=UINT**: File attribute cache timeout in seconds. (default: 1) +* **cache.entry=UINT**: File name lookup cache timeout in seconds. (default: 1) +* **cache.negative_entry=UINT**: Negative file name lookup cache timeout in seconds. (default: 0) * **cache.files=libfuse|off|partial|full|auto-full**: File page caching mode (default: libfuse) * **cache.writeback=BOOL**: Enable kernel writeback caching (default: false) * **cache.symlinks=BOOL**: Cache symlinks (if supported by kernel) (default: false) @@ -139,17 +144,18 @@ See the mergerfs [wiki for real world deployments](https://github.com/trapexit/m #### Value Types * BOOL = 'true' | 'false' -* INT = [0,MAX_INT] +* INT = [MIN_INT,MAX_INT] +* UINT = [0,MAX_INT] * SIZE = 'NNM'; NN = INT, M = 'K' | 'M' | 'G' | 'T' * STR = string -* FUNC = FUSE function -* CATEGORY = FUSE function category +* FUNC = filesystem function +* CATEGORY = function category * POLICY = mergerfs function policy ### branches -The 'branches' (formerly 'srcmounts') argument is a colon (':') delimited list of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same device and any features which aren't supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors. +The 'branches' argument is a colon (':') delimited list of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same device and any features which aren't supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors. Branches currently have two options which can be set. A type which impacts whether or not the branch is included in a policy calculation and a individual minfreespace value. The values are set by prepending an `=` at the end of a branch designation and using commas as delimiters. Example: /mnt/drive=RW,1234 @@ -270,25 +276,47 @@ This hack addresses the issue where the creation of a file with a read-only mode Even though it's a more niche situation this hack breaks normal security and behavior and as such is `off` by default. If set to `git` it will only perform the hack when the path in question includes `/.git/`. `all` will result it it applying anytime a readonly file which is empty is opened for writing. -# FUNCTIONS / POLICIES / CATEGORIES +# FUNCTIONS, CATEGORIES and POLICIES -The POSIX filesystem API is made up of a number of functions. **creat**, **stat**, **chown**, etc. For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: **action**, **create**, and **search**. These functions and categories can be assigned a policy which dictates which underlying branch/file/directory is chosen when performing that behavior. Any policy can be assigned to a function or category though some may not be very useful in practice. For instance: **rand** (random) may be useful for file creation (create) but could lead to very odd behavior if used for `chmod` if there were more than one copy of the file. +The POSIX filesystem API is made up of a number of functions. **creat**, **stat**, **chown**, etc. For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: **action**, **create**, and **search**. These functions and categories can be assigned a policy which dictates which branch is chosen when performing that function. -Some functions, listed in the category `N/A` below, can not be assigned the normal policies. All functions which work on file handles use the handle which was acquired by `open` or `create`. `readdir` has no real need for a policy given the purpose is merely to return a list of entries in a directory. `statfs`'s behavior can be modified via other options. That said many times the current FUSE kernel driver will not always provide the file handle when a client calls `fgetattr`, `fchown`, `fchmod`, `futimens`, `ftruncate`, etc. This means it will call the regular, path based, versions. +Some functions, listed in the category `N/A` below, can not be assigned the normal policies. These functions work with file handles, rather than file paths, which were created by `open` or `create`. That said many times the current FUSE kernel driver will not always provide the file handle when a client calls `fgetattr`, `fchown`, `fchmod`, `futimens`, `ftruncate`, etc. This means it will call the regular, path based, versions. `readdir` has no real need for a policy given the purpose is merely to return a list of entries in a directory. `statfs`'s behavior can be modified via other options. -When using policies which are based on a branch's available space the base path provided is used. Not the full path to the file in question. Meaning that sub mounts won't be considered in the space calculations. The reason is that it doesn't really work for non-path preserving policies and can lead to non-obvious behaviors. +When using policies which are based on a branch's available space the base path provided is used. Not the full path to the file in question. Meaning that mounts in the branch won't be considered in the space calculations. The reason is that it doesn't really work for non-path preserving policies and can lead to non-obvious behaviors. +NOTE: While any policy can be assigned to a function or category though some may not be very useful in practice. For instance: **rand** (random) may be useful for file creation (create) but could lead to very odd behavior if used for `chmod` if there were more than one copy of the file. -#### Functions and their Category classifications + +### Functions and their Category classifications | Category | FUSE Functions | |----------|-------------------------------------------------------------------------------------| | action | chmod, chown, link, removexattr, rename, rmdir, setxattr, truncate, unlink, utimens | | create | create, mkdir, mknod, symlink | -| search | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink | +| search | access, getattr, getxattr, ioctl (directories), listxattr, open, readlink | | N/A | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write, copy_file_range | -In cases where something may be searched (to confirm a directory exists across all source mounts) **getattr** will be used. +In cases where something may be searched for (such as a path to clone) **getattr** will usually be used. + + +### Policies + +A policy is the algorithm used to choose a branch or branches for a function to work on. Think of them as ways to filter and sort branches. + +Any function in the `create` category will clone the relative path if needed. Some other functions (`rename`,`link`,`ioctl`) have special requirements or behaviors which you can read more about below. + + +#### Filtering + +Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting the branches. Filters include **minfreespace**, whether or not a branch is mounted read-only, and the branch tagging (RO,NC,RW). These filters are applied across all policies unless otherwise noted. + +* No **search** function policies filter. +* All **action** function policies filter out branches which are mounted **read-only** or tagged as **RO (read-only)**. +* All **create** function policies filter out branches which are mounted **read-only**, tagged **RO (read-only)** or **NC (no create)**, or has available space less than `minfreespace`. + +Policies may have their own additional filtering such as those that require existing paths to be present. + +If all branches are filtered an error will be returned. Typically **EROFS** (read-only filesystem) or **ENOSPC** (no space left on device) depending on the most recent reason for filtering a branch. **ENOENT** will be returned if no elegible branch is found. #### Path Preservation @@ -304,20 +332,9 @@ When using non-path preserving policies paths will be cloned to target drives as With the `msp` or `most shared path` policies they are defined as `path preserving` for the purpose of controlling `link` and `rename`'s behaviors since `ignorepponrename` is available to disable that behavior. In mergerfs v3.0 the path preserving behavior of rename and link will likely be separated from the policy all together. -#### Filters - -Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting. Filters include **minfreespace**, whether or not a branch is mounted read-only, and the branch tagging (RO,NC,RW). The policy defines the sorting but filtering is mostly uniform as described below. - -* No **search** policies filter. -* All **action** policies will filter out branches which are mounted **read-only** or tagged as **RO (read-only)**. -* All **create** policies will filter out branches which are mounted **read-only**, tagged **RO (read-only)** or **NC (no create)**, or has available space less than `minfreespace`. - -If all branches are filtered an error will be returned. Typically **EROFS** (read-only filesystem) or **ENOSPC** (no space left on device) depending on the most recent reason for filtering a branch. - - #### Policy descriptions -Because of the nature of the behavior the policies act differently depending on the function it is used with (based on the category). +A policy's behavior differs, as mentioned above, based on the function it is used with. Sometimes it really might not make sense to even offer certain policies because they are literally the same as others but it makes things a bit more uniform. In mergerfs 3.0 this might change. | Policy | Description | @@ -360,19 +377,6 @@ Because of the nature of the behavior the policies act differently depending on When `ioctl` is used with an open file then it will use the file handle which was created at the original `open` call. However, when using `ioctl` with a directory mergerfs will use the `open` policy to find the directory to act on. -#### unlink - -In FUSE there is an opaque "file handle" which is created by `open`, `create`, or `opendir`, passed to the kernel, and then is passed back to the FUSE userland application by the kernel. Unfortunately, the FUSE kernel driver does not always send the file handle when it theoretically could/should. This complicates certain behaviors / workflows particularly in the high level API. As a result mergerfs is currently doing a few hacky things. - -libfuse2 and libfuse3, when using the high level API, will rename names to `.fuse_hiddenXXXXXX` if the file is open when unlinked or renamed over. It does this so the file is still available when a request referencing the now missing file is made. This file however keeps a `rmdir` from succeeding and can be picked up by software reading directories. - -The change mergerfs has done is that if a file is open when an unlink or rename happens it will open the file and keep it open till closed by all those who opened it prior. When a request comes in referencing that file and it doesn't include a file handle it will instead use the file handle created at unlink/rename time. - -This won't result in technically proper behavior but close enough for many usecases. - -The plan is to rewrite mergerfs to use the low level API so these invasive libfuse changes are no longer necessary. - - #### rename & link #### **NOTE:** If you're receiving errors from software when files are moved / renamed / linked then you should consider changing the create policy to one which is **not** path preserving, enabling `ignorepponrename`, or contacting the author of the offending software and requesting that `EXDEV` (cross device / improper link) be properly handled. @@ -473,7 +477,7 @@ $ wget https://github.com/trapexit/mergerfs/releases/download//mergerfs-_.deb ``` #### RHEL / CentOS /Fedora @@ -664,7 +668,6 @@ A B C * mergerfs.dup: Ensure there are at least N copies of a file across the pool * mergerfs.balance: Rebalance files across drives by moving them from the most filled to the least filled * mergerfs.consolidate: move files within a single mergerfs directory to the drive with most free space - * mergerfs.mktrash: Creates FreeDesktop.org Trash specification compatible directories on a mergerfs mount * https://github.com/trapexit/scorch * scorch: A tool to help discover silent corruption of files and keep track of files * https://github.com/trapexit/bbf @@ -869,6 +872,8 @@ $ dd if=/mnt/mergerfs/1GB.file of=/dev/null bs=1M count=1024 iflag=nocache conv= # TIPS / NOTES +* This document is very literal and thorough. Unless there is a bug things work as described. If a suspected feature isn't mentioned it doesn't exist. +* Ensure you're using the latest version. Few distros have the latest version. * **use_ino** will only work when used with mergerfs 2.18.0 and above. * Run mergerfs as `root` (with **allow_other**) unless you're merging paths which are owned by the same user otherwise strange permission issues may arise. * https://github.com/trapexit/backup-and-recovery-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure. @@ -885,6 +890,7 @@ $ dd if=/mnt/mergerfs/1GB.file of=/dev/null bs=1M count=1024 iflag=nocache conv= [https://github.com/trapexit/mergerfs/wiki/Kernel-Issues-&-Bugs](https://github.com/trapexit/mergerfs/wiki/Kernel-Issues-&-Bugs) + #### directory mtime is not being updated Remember that the default policy for `getattr` is `ff`. The information for the first directory found will be returned. If it wasn't the directory which had been updated then it will appear outdated. diff --git a/man/mergerfs.1 b/man/mergerfs.1 index 62682465..9f79e2f5 100644 --- a/man/mergerfs.1 +++ b/man/mergerfs.1 @@ -42,7 +42,7 @@ Can turn read\-only files into symlinks to underlying file .IP \[bu] 2 Hard link copy\-on\-write / CoW .IP \[bu] 2 -supports POSIX ACLs +Support for POSIX ACLs .SH HOW IT WORKS .PP mergerfs logically merges multiple paths together. @@ -74,14 +74,14 @@ A\ \ \ \ \ \ \ \ \ +\ \ \ \ \ \ B\ \ \ \ \ \ \ \ =\ \ \ \ \ \ \ C \f[] .fi .PP -mergerfs does \f[B]not\f[] support the copy\-on\-write (CoW) behavior -found in \f[B]aufs\f[] and \f[B]overlayfs\f[]. +mergerfs does \f[B]NOT\f[] support the copy\-on\-write (CoW) or whiteout +behaviors found in \f[B]aufs\f[] and \f[B]overlayfs\f[]. You can \f[B]not\f[] mount a read\-only filesystem and write to it. However, mergerfs will ignore read\-only drives when creating new files so you can mix read\-write and read\-only drives. -It also does \f[B]not\f[] split data across drives. +It also does \f[B]NOT\f[] split data across drives. It is not RAID0 / striping. -It is simply a union. +It is simply a union of other filesystems. .SH TERMINOLOGY .IP \[bu] 2 branch: A base path used in the pool. @@ -91,11 +91,12 @@ The union of the branches. .IP \[bu] 2 relative path: The path in the pool relative to the branch and mount. .IP \[bu] 2 -policy: The algorithm used to select a file when performing a function. +function: A filesystem call (open, unlink, create, getattr, rmdir, etc.) .IP \[bu] 2 -function: A filesystem call (open, unlink, create, getattr, etc.) +category: A collection of functions based on basic behavior (action, +create, search). .IP \[bu] 2 -category: A collection of functions (action, create, search). +policy: The algorithm used to select a file when performing a function. .IP \[bu] 2 path preservation: Aspect of some policies which includes checking the path for which a file would be created. @@ -103,21 +104,26 @@ path for which a file would be created. .PP If you don\[aq]t already know that you have a special use case then just start with one of the following option sets. -.SS You don\[aq]t need \f[C]mmap\f[] +.SS You need \f[C]mmap\f[] (used by rtorrent and many sqlite3 base +software) .PP -\f[C]use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs\f[] -.SS You do need \f[C]mmap\f[] (used by rtorrent and some other programs) +\f[C]allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs\f[] +.SS You don\[aq]t need \f[C]mmap\f[] .PP -\f[C]use_ino,cache.files=partial,dropcacheonclose=true,allow_other,category.create=mfs\f[] +\f[C]allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs\f[] .PP See the mergerfs wiki for real world deployments (https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments) for comparisons / ideas. .SH OPTIONS +.PP +These options are the same regardless you use them with the +\f[C]mergerfs\f[] commandline program, used in fstab, or in a config +file. .SS mount options .IP \[bu] 2 \f[B]config\f[]: Path to a config file. -Same arguments as below in key=val format. +Same arguments as below in key=val / ini style format. .IP \[bu] 2 \f[B]branches\f[]: Colon delimited list of branches. .IP \[bu] 2 @@ -163,7 +169,7 @@ be reported as symlinks to the original files. Please read more below before using. (default: false) .IP \[bu] 2 -\f[B]symlinkify_timeout=INT\f[]: Time to wait, in seconds, to activate +\f[B]symlinkify_timeout=UINT\f[]: Time to wait, in seconds, to activate the \f[B]symlinkify\f[] behavior. (default: 3600) .IP \[bu] 2 @@ -227,7 +233,7 @@ pending read request per file handle and will attempt to order requests by offset. (default: true) .IP \[bu] 2 -\f[B]fuse_msg_size=INT\f[]: Set the max number of pages per FUSE +\f[B]fuse_msg_size=UINT\f[]: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256) @@ -254,24 +260,32 @@ longest common prefix removed. See below for the list of value types. Example: \f[B]func.getattr=newest\f[] .IP \[bu] 2 -\f[B]category.CATEGORY=POLICY\f[]: Sets policy of all FUSE functions in -the provided category. -See POLICIES section for defaults. -Example: \f[B]category.create=mfs\f[] +\f[B]category.action=POLICY\f[]: Sets policy of all FUSE functions in +the action category. +(default: epall) +.IP \[bu] 2 +\f[B]category.create=POLICY\f[]: Sets policy of all FUSE functions in +the create category. +(default: epmfs) .IP \[bu] 2 -\f[B]cache.open=INT\f[]: \[aq]open\[aq] policy cache timeout in seconds. +\f[B]category.search=POLICY\f[]: Sets policy of all FUSE functions in +the search category. +(default: ff) +.IP \[bu] 2 +\f[B]cache.open=UINT\f[]: \[aq]open\[aq] policy cache timeout in +seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.statfs=INT\f[]: \[aq]statfs\[aq] cache timeout in seconds. +\f[B]cache.statfs=UINT\f[]: \[aq]statfs\[aq] cache timeout in seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.attr=INT\f[]: File attribute cache timeout in seconds. +\f[B]cache.attr=UINT\f[]: File attribute cache timeout in seconds. (default: 1) .IP \[bu] 2 -\f[B]cache.entry=INT\f[]: File name lookup cache timeout in seconds. +\f[B]cache.entry=UINT\f[]: File name lookup cache timeout in seconds. (default: 1) .IP \[bu] 2 -\f[B]cache.negative_entry=INT\f[]: Negative file name lookup cache +\f[B]cache.negative_entry=UINT\f[]: Negative file name lookup cache timeout in seconds. (default: 0) .IP \[bu] 2 @@ -315,22 +329,24 @@ setting. .IP \[bu] 2 BOOL = \[aq]true\[aq] | \[aq]false\[aq] .IP \[bu] 2 -INT = [0,MAX_INT] +INT = [MIN_INT,MAX_INT] +.IP \[bu] 2 +UINT = [0,MAX_INT] .IP \[bu] 2 SIZE = \[aq]NNM\[aq]; NN = INT, M = \[aq]K\[aq] | \[aq]M\[aq] | \[aq]G\[aq] | \[aq]T\[aq] .IP \[bu] 2 STR = string .IP \[bu] 2 -FUNC = FUSE function +FUNC = filesystem function .IP \[bu] 2 -CATEGORY = FUSE function category +CATEGORY = function category .IP \[bu] 2 POLICY = mergerfs function policy .SS branches .PP -The \[aq]branches\[aq] (formerly \[aq]srcmounts\[aq]) argument is a -colon (\[aq]:\[aq]) delimited list of paths to be pooled together. +The \[aq]branches\[aq] argument is a colon (\[aq]:\[aq]) delimited list +of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same @@ -613,7 +629,7 @@ If set to \f[C]git\f[] it will only perform the hack when the path in question includes \f[C]/.git/\f[]. \f[C]all\f[] will result it it applying anytime a readonly file which is empty is opened for writing. -.SH FUNCTIONS / POLICIES / CATEGORIES +.SH FUNCTIONS, CATEGORIES and POLICIES .PP The POSIX filesystem API is made up of a number of functions. \f[B]creat\f[], \f[B]stat\f[], \f[B]chown\f[], etc. @@ -621,34 +637,34 @@ For ease of configuration in mergerfs most of the core functions are grouped into 3 categories: \f[B]action\f[], \f[B]create\f[], and \f[B]search\f[]. These functions and categories can be assigned a policy which dictates -which underlying branch/file/directory is chosen when performing that -behavior. -Any policy can be assigned to a function or category though some may not -be very useful in practice. -For instance: \f[B]rand\f[] (random) may be useful for file creation -(create) but could lead to very odd behavior if used for \f[C]chmod\f[] -if there were more than one copy of the file. +which branch is chosen when performing that function. .PP Some functions, listed in the category \f[C]N/A\f[] below, can not be assigned the normal policies. -All functions which work on file handles use the handle which was -acquired by \f[C]open\f[] or \f[C]create\f[]. -\f[C]readdir\f[] has no real need for a policy given the purpose is -merely to return a list of entries in a directory. -\f[C]statfs\f[]\[aq]s behavior can be modified via other options. +These functions work with file handles, rather than file paths, which +were created by \f[C]open\f[] or \f[C]create\f[]. That said many times the current FUSE kernel driver will not always provide the file handle when a client calls \f[C]fgetattr\f[], \f[C]fchown\f[], \f[C]fchmod\f[], \f[C]futimens\f[], \f[C]ftruncate\f[], etc. This means it will call the regular, path based, versions. +\f[C]readdir\f[] has no real need for a policy given the purpose is +merely to return a list of entries in a directory. +\f[C]statfs\f[]\[aq]s behavior can be modified via other options. .PP When using policies which are based on a branch\[aq]s available space the base path provided is used. Not the full path to the file in question. -Meaning that sub mounts won\[aq]t be considered in the space +Meaning that mounts in the branch won\[aq]t be considered in the space calculations. The reason is that it doesn\[aq]t really work for non\-path preserving policies and can lead to non\-obvious behaviors. +.PP +NOTE: While any policy can be assigned to a function or category though +some may not be very useful in practice. +For instance: \f[B]rand\f[] (random) may be useful for file creation +(create) but could lead to very odd behavior if used for \f[C]chmod\f[] +if there were more than one copy of the file. .SS Functions and their Category classifications .PP .TS @@ -685,8 +701,44 @@ fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl T} .TE .PP -In cases where something may be searched (to confirm a directory exists -across all source mounts) \f[B]getattr\f[] will be used. +In cases where something may be searched for (such as a path to clone) +\f[B]getattr\f[] will usually be used. +.SS Policies +.PP +A policy is the algorithm used to choose a branch or branches for a +function to work on. +Think of them as ways to filter and sort branches. +.PP +Any function in the \f[C]create\f[] category will clone the relative +path if needed. +Some other functions (\f[C]rename\f[],\f[C]link\f[],\f[C]ioctl\f[]) have +special requirements or behaviors which you can read more about below. +.SS Filtering +.PP +Policies basically search branches and create a list of files / paths +for functions to work on. +The policy is responsible for filtering and sorting the branches. +Filters include \f[B]minfreespace\f[], whether or not a branch is +mounted read\-only, and the branch tagging (RO,NC,RW). +These filters are applied across all policies unless otherwise noted. +.IP \[bu] 2 +No \f[B]search\f[] function policies filter. +.IP \[bu] 2 +All \f[B]action\f[] function policies filter out branches which are +mounted \f[B]read\-only\f[] or tagged as \f[B]RO (read\-only)\f[]. +.IP \[bu] 2 +All \f[B]create\f[] function policies filter out branches which are +mounted \f[B]read\-only\f[], tagged \f[B]RO (read\-only)\f[] or \f[B]NC +(no create)\f[], or has available space less than \f[C]minfreespace\f[]. +.PP +Policies may have their own additional filtering such as those that +require existing paths to be present. +.PP +If all branches are filtered an error will be returned. +Typically \f[B]EROFS\f[] (read\-only filesystem) or \f[B]ENOSPC\f[] (no +space left on device) depending on the most recent reason for filtering +a branch. +\f[B]ENOENT\f[] will be returned if no elegible branch is found. .SS Path Preservation .PP Policies, as described below, are of two basic types. @@ -709,33 +761,14 @@ defined as \f[C]path\ preserving\f[] for the purpose of controlling \f[C]ignorepponrename\f[] is available to disable that behavior. In mergerfs v3.0 the path preserving behavior of rename and link will likely be separated from the policy all together. -.SS Filters -.PP -Policies basically search branches and create a list of files / paths -for functions to work on. -The policy is responsible for filtering and sorting. -Filters include \f[B]minfreespace\f[], whether or not a branch is -mounted read\-only, and the branch tagging (RO,NC,RW). -The policy defines the sorting but filtering is mostly uniform as -described below. -.IP \[bu] 2 -No \f[B]search\f[] policies filter. -.IP \[bu] 2 -All \f[B]action\f[] policies will filter out branches which are mounted -\f[B]read\-only\f[] or tagged as \f[B]RO (read\-only)\f[]. -.IP \[bu] 2 -All \f[B]create\f[] policies will filter out branches which are mounted -\f[B]read\-only\f[], tagged \f[B]RO (read\-only)\f[] or \f[B]NC (no -create)\f[], or has available space less than \f[C]minfreespace\f[]. -.PP -If all branches are filtered an error will be returned. -Typically \f[B]EROFS\f[] (read\-only filesystem) or \f[B]ENOSPC\f[] (no -space left on device) depending on the most recent reason for filtering -a branch. .SS Policy descriptions .PP -Because of the nature of the behavior the policies act differently -depending on the function it is used with (based on the category). +A policy\[aq]s behavior differs, as mentioned above, based on the +function it is used with. +Sometimes it really might not make sense to even offer certain policies +because they are literally the same as others but it makes things a bit +more uniform. +In mergerfs 3.0 this might change. .PP .TS tab(@); @@ -930,38 +963,6 @@ When \f[C]ioctl\f[] is used with an open file then it will use the file handle which was created at the original \f[C]open\f[] call. However, when using \f[C]ioctl\f[] with a directory mergerfs will use the \f[C]open\f[] policy to find the directory to act on. -.SS unlink -.PP -In FUSE there is an opaque "file handle" which is created by -\f[C]open\f[], \f[C]create\f[], or \f[C]opendir\f[], passed to the -kernel, and then is passed back to the FUSE userland application by the -kernel. -Unfortunately, the FUSE kernel driver does not always send the file -handle when it theoretically could/should. -This complicates certain behaviors / workflows particularly in the high -level API. -As a result mergerfs is currently doing a few hacky things. -.PP -libfuse2 and libfuse3, when using the high level API, will rename names -to \f[C]\&.fuse_hiddenXXXXXX\f[] if the file is open when unlinked or -renamed over. -It does this so the file is still available when a request referencing -the now missing file is made. -This file however keeps a \f[C]rmdir\f[] from succeeding and can be -picked up by software reading directories. -.PP -The change mergerfs has done is that if a file is open when an unlink or -rename happens it will open the file and keep it open till closed by all -those who opened it prior. -When a request comes in referencing that file and it doesn\[aq]t include -a file handle it will instead use the file handle created at -unlink/rename time. -.PP -This won\[aq]t result in technically proper behavior but close enough -for many usecases. -.PP -The plan is to rewrite mergerfs to use the low level API so these -invasive libfuse changes are no longer necessary. .SS rename & link .PP \f[B]NOTE:\f[] If you\[aq]re receiving errors from software when files @@ -1152,7 +1153,7 @@ $\ wget\ https://github.com/trapexit/mergerfs/releases/download//mergerfs\- $\ cd\ mergerfs $\ sudo\ tools/install\-build\-pkgs $\ make\ deb -$\ sudo\ dpkg\ \-i\ ../mergerfs_version_arch.deb +$\ sudo\ dpkg\ \-i\ ../mergerfs__.deb \f[] .fi .SS RHEL / CentOS /Fedora @@ -1449,9 +1450,6 @@ most filled to the least filled mergerfs.consolidate: move files within a single mergerfs directory to the drive with most free space .IP \[bu] 2 -mergerfs.mktrash: Creates FreeDesktop.org Trash specification compatible -directories on a mergerfs mount -.IP \[bu] 2 https://github.com/trapexit/scorch .IP \[bu] 2 scorch: A tool to help discover silent corruption of files and keep @@ -1741,7 +1739,7 @@ do \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ head\ \-n\ 1\ |\ \\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ cut\ \-d\[aq]\ \[aq]\ \-f2\-) \ \ \ \ test\ \-n\ "${FILE}" -\ \ \ \ rsync\ \-axqHAXWES\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/./${FILE}"\ "${BACKING}/" +\ \ \ \ rsync\ \-axqHAXWESR\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/./${FILE}"\ "${BACKING}/" done \f[] .fi @@ -1893,6 +1891,13 @@ $\ dd\ if=/mnt/mergerfs/1GB.file\ of=/dev/null\ bs=1M\ count=1024\ iflag=nocache .fi .SH TIPS / NOTES .IP \[bu] 2 +This document is very literal and thorough. +Unless there is a bug things work as described. +If a suspected feature isn\[aq]t mentioned it doesn\[aq]t exist. +.IP \[bu] 2 +Ensure you\[aq]re using the latest version. +Few distros have the latest version. +.IP \[bu] 2 \f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and above. .IP \[bu] 2 From 3808e2683ea9cd3c79f340c7eba071a49920855b Mon Sep 17 00:00:00 2001 From: trapexit Date: Fri, 1 Jan 2021 15:52:15 -0500 Subject: [PATCH 2/6] Update FUNDING.yml --- .github/FUNDING.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml index ee3f78b5..e44044e8 100644 --- a/.github/FUNDING.yml +++ b/.github/FUNDING.yml @@ -2,6 +2,6 @@ github: [trapexit] patreon: trapexit -open_collective: trapexit +custom: ['https://paypal.me/trapexit','https://buymeacoff.ee/trapexit','https://www.subscribestar.com/trapexit'] ko_fi: trapexit -custom: ['https://paypal.me/trapexit','https://www.subscribestar.com/trapexit'] +open_collective: trapexit From 538467b86dd17358fe98e6f001dfe000cf27f920 Mon Sep 17 00:00:00 2001 From: Antonio SJ Musumeci Date: Sat, 19 Dec 2020 15:06:08 -0500 Subject: [PATCH 3/6] config: rework global config, remove rwlock, make branches RCU like Also added unit tests. Should have done separately but found a number of bugs. --- LICENSE | 2 +- Makefile | 32 +- src/branch.cpp | 366 +---- src/branch.hpp | 67 +- src/branches.cpp | 429 ++++++ src/branches.hpp | 94 ++ src/category.cpp | 49 + src/category.hpp | 93 +- src/config.cpp | 228 ++- src/config.hpp | 98 +- src/config_cachefiles.hpp | 1 + src/config_inodecalc.cpp | 1 + src/config_inodecalc.hpp | 5 +- src/config_moveonenospc.cpp | 9 +- src/config_moveonenospc.hpp | 14 +- src/config_nfsopenhack.cpp | 1 + src/config_nfsopenhack.hpp | 1 + src/config_readdir.cpp | 1 + src/config_readdir.hpp | 1 + src/config_statfs.cpp | 1 + src/config_statfs.hpp | 1 + src/config_statfsignore.cpp | 1 + src/config_xattr.cpp | 1 + src/config_xattr.hpp | 1 + src/dirinfo.hpp | 1 + src/endian.hpp | 1 + src/enum.hpp | 1 + src/fh.hpp | 1 + src/fileinfo.hpp | 1 + src/fixed_mem_pool.hpp | 4 +- src/from_string.cpp | 17 +- src/from_string.hpp | 2 +- src/fs_acl.cpp | 1 + src/fs_acl.hpp | 1 + src/fs_attr.hpp | 1 + src/fs_attr_linux.icpp | 1 + src/fs_attr_unsupported.icpp | 1 + src/fs_clonefile.cpp | 1 + src/fs_clonefile.hpp | 1 + src/fs_clonepath.cpp | 5 +- src/fs_clonepath.hpp | 1 + src/fs_close.hpp | 1 + src/fs_closedir.hpp | 1 + src/fs_copy_file_range.hpp | 4 +- src/fs_copy_file_range_linux.icpp | 4 +- src/fs_copy_file_range_unsupported.icpp | 4 +- src/fs_copydata.cpp | 1 + src/fs_copydata.hpp | 1 + src/fs_copydata_copy_file_range.cpp | 3 +- src/fs_copydata_copy_file_range.hpp | 3 +- src/fs_copydata_readwrite.cpp | 1 + src/fs_copydata_readwrite.hpp | 1 + src/fs_cow.cpp | 1 + src/fs_cow.hpp | 1 + src/fs_devid.hpp | 1 + src/fs_dirfd.hpp | 1 + src/fs_dup.hpp | 1 + src/fs_eaccess.hpp | 1 + src/fs_exists.hpp | 1 + src/fs_faccessat.hpp | 1 + src/fs_fadvise.hpp | 1 + src/fs_fadvise_posix.icpp | 1 + src/fs_fadvise_unsupported.icpp | 1 + src/fs_fallocate.hpp | 1 + src/fs_fallocate_linux.icpp | 1 + src/fs_fallocate_osx.icpp | 3 +- src/fs_fallocate_posix.icpp | 1 + src/fs_fallocate_unsupported.icpp | 1 + src/fs_fchmod.hpp | 1 + src/fs_fchmodat.hpp | 1 + src/fs_fchown.hpp | 1 + src/fs_fdatasync.hpp | 1 + src/fs_fgetxattr.hpp | 1 + src/fs_ficlone.hpp | 1 + src/fs_ficlone_linux.icpp | 1 + src/fs_ficlone_unsupported.icpp | 1 + src/fs_file_size.cpp | 3 +- src/fs_file_size.hpp | 3 +- src/fs_findallfiles.cpp | 1 + src/fs_findallfiles.hpp | 9 +- src/fs_findonfs.cpp | 32 +- src/fs_findonfs.hpp | 11 +- src/fs_flistxattr.hpp | 1 + src/fs_flock.hpp | 1 + src/fs_fsetxattr.hpp | 1 + src/fs_fstat.hpp | 1 + src/fs_fstatat.hpp | 1 + src/fs_fsync.hpp | 1 + src/fs_ftruncate.hpp | 1 + src/fs_futimens.hpp | 1 + src/fs_futimens_freebsd_11.hpp | 1 + src/fs_futimens_generic.hpp | 1 + src/fs_futimens_linux.hpp | 1 + src/fs_futimesat.hpp | 1 + src/fs_futimesat_generic.icpp | 1 + src/fs_futimesat_osx.icpp | 1 + src/fs_getdents64.cpp | 1 + src/fs_getdents64.hpp | 1 + src/fs_getfl.cpp | 1 + src/fs_getfl.hpp | 1 + src/fs_glob.cpp | 4 +- src/fs_glob.hpp | 1 + src/fs_has_space.cpp | 2 +- src/fs_has_space.hpp | 2 +- src/fs_info.cpp | 4 +- src/fs_info.hpp | 1 + src/fs_info_t.hpp | 3 +- src/fs_inode.cpp | 8 +- src/fs_inode.hpp | 2 +- src/fs_ioctl.hpp | 1 + src/fs_lchmod.hpp | 1 + src/fs_lchown.hpp | 1 + src/fs_lgetxattr.hpp | 1 + src/fs_link.hpp | 1 + src/fs_llistxattr.hpp | 1 + src/fs_lremovexattr.hpp | 1 + src/fs_lseek.hpp | 1 + src/fs_lsetxattr.hpp | 1 + src/fs_lstat.hpp | 1 + src/fs_lutimens.hpp | 1 + src/fs_mkdir.hpp | 1 + src/fs_mknod.hpp | 1 + src/fs_mktemp.cpp | 1 + src/fs_mktemp.hpp | 1 + src/fs_movefile.cpp | 21 +- src/fs_movefile.hpp | 19 +- src/fs_open.hpp | 1 + src/fs_opendir.hpp | 1 + src/fs_path.cpp | 9 +- src/fs_path.hpp | 1 + src/fs_read.hpp | 1 + src/fs_readdir.hpp | 1 + src/fs_readlink.hpp | 1 + src/fs_realpath.hpp | 1 + src/fs_realpathize.cpp | 1 + src/fs_realpathize.hpp | 1 + src/fs_remove.hpp | 1 + src/fs_rename.hpp | 1 + src/fs_rmdir.hpp | 1 + src/fs_sendfile.hpp | 1 + src/fs_sendfile_linux.icpp | 3 +- src/fs_sendfile_unsupported.icpp | 3 +- src/fs_setfl.cpp | 1 + src/fs_setfl.hpp | 1 + src/fs_stat.hpp | 1 + src/fs_stat_utils.hpp | 1 + src/fs_statvfs.hpp | 2 + src/fs_statvfs_cache.cpp | 3 +- src/fs_statvfs_cache.hpp | 4 +- src/fs_symlink.hpp | 1 + src/fs_truncate.hpp | 1 + src/fs_unlink.hpp | 1 + src/fs_utimensat_freebsd.hpp | 1 + src/fs_utimensat_generic.hpp | 1 + src/fs_utimensat_linux.hpp | 1 + src/fs_write.hpp | 1 + src/fs_xattr.cpp | 2 +- src/fs_xattr.hpp | 1 + src/func.cpp | 39 +- src/func.hpp | 267 ++-- src/func_category.cpp | 90 -- src/func_category.hpp | 69 - src/funcs.hpp | 44 +- src/fuse_access.cpp | 31 +- src/fuse_access.hpp | 1 + src/fuse_chmod.cpp | 26 +- src/fuse_chmod.hpp | 1 + src/fuse_chown.cpp | 16 +- src/fuse_chown.hpp | 1 + src/fuse_copy_file_range.cpp | 3 +- src/fuse_copy_file_range.hpp | 1 + src/fuse_create.cpp | 35 +- src/fuse_create.hpp | 3 +- src/fuse_destroy.hpp | 1 + src/fuse_fallocate.cpp | 3 +- src/fuse_fallocate.hpp | 1 + src/fuse_fchmod.cpp | 3 +- src/fuse_fchmod.hpp | 3 +- src/fuse_fchown.cpp | 3 +- src/fuse_fchown.hpp | 3 +- src/fuse_fgetattr.cpp | 11 +- src/fuse_fgetattr.hpp | 3 +- src/fuse_flock.cpp | 3 +- src/fuse_flock.hpp | 3 +- src/fuse_flush.cpp | 3 +- src/fuse_flush.hpp | 3 +- src/fuse_free_hide.cpp | 3 +- src/fuse_free_hide.hpp | 3 +- src/fuse_fsync.cpp | 3 +- src/fuse_fsync.hpp | 5 +- src/fuse_fsyncdir.cpp | 3 +- src/fuse_fsyncdir.hpp | 3 +- src/fuse_ftruncate.cpp | 3 +- src/fuse_ftruncate.hpp | 3 +- src/fuse_futimens.cpp | 3 +- src/fuse_futimens.hpp | 3 +- src/fuse_getattr.cpp | 46 +- src/fuse_getattr.hpp | 1 + src/fuse_getxattr.cpp | 47 +- src/fuse_getxattr.hpp | 1 + src/fuse_init.cpp | 25 +- src/fuse_init.hpp | 3 +- src/fuse_ioctl.cpp | 58 +- src/fuse_ioctl.hpp | 5 +- src/fuse_link.cpp | 65 +- src/fuse_link.hpp | 1 + src/fuse_listxattr.cpp | 26 +- src/fuse_listxattr.hpp | 1 + src/fuse_mkdir.cpp | 40 +- src/fuse_mkdir.hpp | 1 + src/fuse_mknod.cpp | 17 +- src/fuse_mknod.hpp | 1 + src/fuse_open.cpp | 38 +- src/fuse_open.hpp | 3 +- src/fuse_opendir.cpp | 7 +- src/fuse_opendir.hpp | 3 +- src/fuse_prepare_hide.cpp | 3 +- src/fuse_prepare_hide.hpp | 3 +- src/fuse_read.cpp | 3 +- src/fuse_read.hpp | 3 +- src/fuse_read_buf.cpp | 3 +- src/fuse_read_buf.hpp | 3 +- src/fuse_readdir.cpp | 15 +- src/fuse_readdir.hpp | 3 +- src/fuse_readdir_linux.cpp | 36 +- src/fuse_readdir_linux.hpp | 13 +- src/fuse_readdir_plus.cpp | 23 +- src/fuse_readdir_plus.hpp | 3 +- src/fuse_readdir_plus_linux.cpp | 46 +- src/fuse_readdir_plus_linux.hpp | 17 +- src/fuse_readdir_plus_posix.cpp | 50 +- src/fuse_readdir_plus_posix.hpp | 17 +- src/fuse_readdir_posix.cpp | 37 +- src/fuse_readdir_posix.hpp | 13 +- src/fuse_readlink.cpp | 20 +- src/fuse_readlink.hpp | 1 + src/fuse_release.cpp | 10 +- src/fuse_release.hpp | 3 +- src/fuse_releasedir.cpp | 3 +- src/fuse_releasedir.hpp | 3 +- src/fuse_removexattr.cpp | 21 +- src/fuse_removexattr.hpp | 1 + src/fuse_rename.cpp | 108 +- src/fuse_rename.hpp | 1 + src/fuse_rmdir.cpp | 16 +- src/fuse_rmdir.hpp | 1 + src/fuse_setxattr.cpp | 78 +- src/fuse_setxattr.hpp | 1 + src/fuse_statfs.cpp | 40 +- src/fuse_statfs.hpp | 1 + src/fuse_symlink.cpp | 31 +- src/fuse_symlink.hpp | 1 + src/fuse_truncate.cpp | 28 +- src/fuse_truncate.hpp | 1 + src/fuse_unlink.cpp | 16 +- src/fuse_unlink.hpp | 1 + src/fuse_utimens.cpp | 32 +- src/fuse_utimens.hpp | 1 + src/fuse_write.cpp | 13 +- src/fuse_write.hpp | 3 +- src/fuse_write_buf.cpp | 9 +- src/fuse_write_buf.hpp | 3 +- src/hw_cpu.cpp | 1 + src/hw_cpu.hpp | 1 + src/mergerfs.cpp | 25 +- src/num.cpp | 3 +- src/num.hpp | 5 +- src/option_parser.cpp | 190 +-- src/option_parser.hpp | 9 +- src/policies.cpp | 132 ++ src/policies.hpp | 121 ++ src/policy.cpp | 96 -- src/policy.hpp | 360 ++--- src/policy_all.cpp | 56 +- src/policy_all.hpp | 61 + src/policy_cache.cpp | 22 +- src/policy_cache.hpp | 12 +- src/policy_epall.cpp | 125 +- src/policy_epall.hpp | 64 + src/policy_epff.cpp | 121 +- src/policy_epff.hpp | 61 + src/policy_eplfs.cpp | 123 +- src/policy_eplfs.hpp | 61 + src/policy_eplus.cpp | 123 +- src/policy_eplus.hpp | 61 + src/policy_epmfs.cpp | 126 +- src/policy_epmfs.hpp | 64 + src/policy_eppfrd.cpp | 161 +- src/policy_eppfrd.hpp | 61 + src/policy_eprand.cpp | 48 +- src/policy_eprand.hpp | 61 + src/policy_erofs.cpp | 27 +- src/policy_erofs.hpp | 61 + src/policy_ff.cpp | 56 +- src/policy_ff.hpp | 61 + src/policy_lfs.cpp | 56 +- src/policy_lfs.hpp | 61 + src/policy_lus.cpp | 55 +- src/policy_lus.hpp | 61 + src/policy_mfs.cpp | 55 +- src/policy_mfs.hpp | 61 + src/policy_msplfs.cpp | 68 +- src/policy_msplfs.hpp | 61 + src/policy_msplus.cpp | 68 +- src/policy_msplus.hpp | 61 + src/policy_mspmfs.cpp | 65 +- src/policy_mspmfs.hpp | 61 + src/policy_msppfrd.cpp | 80 +- src/policy_msppfrd.hpp | 61 + src/policy_newest.cpp | 121 +- src/policy_newest.hpp | 61 + src/policy_pfrd.cpp | 68 +- src/policy_pfrd.hpp | 61 + src/policy_rand.cpp | 48 +- src/policy_rand.hpp | 61 + src/rnd.cpp | 4 +- src/rnd.hpp | 3 +- src/statvfs_util.hpp | 3 +- src/str.cpp | 25 +- src/str.hpp | 5 + src/{policy_invalid.cpp => strvec.hpp} | 19 +- src/to_string.cpp | 2 +- src/to_string.hpp | 2 +- src/tofrom_wrapper.hpp | 15 +- tests/acutest.h | 1794 +++++++++++++++++++++++ tests/tests.cpp | 283 ++++ tools/cppfind | 2 +- 327 files changed, 6646 insertions(+), 2991 deletions(-) create mode 100644 src/branches.cpp create mode 100644 src/branches.hpp create mode 100644 src/category.cpp delete mode 100644 src/func_category.cpp delete mode 100644 src/func_category.hpp create mode 100644 src/policies.cpp create mode 100644 src/policies.hpp delete mode 100644 src/policy.cpp create mode 100644 src/policy_all.hpp create mode 100644 src/policy_epall.hpp create mode 100644 src/policy_epff.hpp create mode 100644 src/policy_eplfs.hpp create mode 100644 src/policy_eplus.hpp create mode 100644 src/policy_epmfs.hpp create mode 100644 src/policy_eppfrd.hpp create mode 100644 src/policy_eprand.hpp create mode 100644 src/policy_erofs.hpp create mode 100644 src/policy_ff.hpp create mode 100644 src/policy_lfs.hpp create mode 100644 src/policy_lus.hpp create mode 100644 src/policy_mfs.hpp create mode 100644 src/policy_msplfs.hpp create mode 100644 src/policy_msplus.hpp create mode 100644 src/policy_mspmfs.hpp create mode 100644 src/policy_msppfrd.hpp create mode 100644 src/policy_newest.hpp create mode 100644 src/policy_pfrd.hpp create mode 100644 src/policy_rand.hpp rename src/{policy_invalid.cpp => strvec.hpp} (67%) create mode 100644 tests/acutest.h create mode 100644 tests/tests.cpp diff --git a/LICENSE b/LICENSE index f867f7d7..b4fdad4f 100644 --- a/LICENSE +++ b/LICENSE @@ -1,7 +1,7 @@ /* ISC License - Copyright (c) 2020, Antonio SJ Musumeci + Copyright (c) 2021, Antonio SJ Musumeci Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above diff --git a/Makefile b/Makefile index d98021c4..ddbc8a2e 100644 --- a/Makefile +++ b/Makefile @@ -58,13 +58,20 @@ LTO_FLAGS := endif SRC = $(wildcard src/*.cpp) -OBJS = $(SRC:src/%.cpp=build/%.o) -DEPS = $(SRC:src/%.cpp=build/%.d) +OBJS = $(SRC:src/%.cpp=build/.src/%.o) +DEPS = $(SRC:src/%.cpp=build/.src/%.d) + +TESTS = $(wildcard tests/*.cpp) +TESTS_OBJS = $(filter-out build/.src/mergerfs.o,$(OBJS)) +TESTS_OBJS += $(TESTS:tests/%.cpp=build/.tests/%.o) +TESTS_DEPS = $(TESTS:tests/%.cpp=build/.tests/%.d) +TESTS_DEPS += $(DEPS) + MANPAGE = mergerfs.1 CXXFLAGS ?= ${OPT_FLAGS} CXXFLAGS := \ ${CXXFLAGS} \ - -std=c++0x \ + -std=c++11 \ $(STATIC_FLAGS) \ $(LTO_FLAGS) \ -Wall \ @@ -77,6 +84,9 @@ FUSE_FLAGS = \ MFS_FLAGS = \ -DUSE_XATTR=$(USE_XATTR) \ -DUGID_USE_RWLOCK=$(UGID_USE_RWLOCK) +TESTS_FLAGS = \ + -Isrc \ + -DTESTS LDFLAGS := \ ${LDFLAGS} \ @@ -110,11 +120,19 @@ help: objects: version build/stamp $(MAKE) $(OBJS) +tests-objects: + $(MAKE) $(TESTS_OBJS) + build/mergerfs: libfuse objects $(CXX) $(CXXFLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(OBJS) -o $@ libfuse/build/libfuse.a $(LDFLAGS) +build/tests: build/mergerfs tests-objects + $(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) $(TESTS_OBJS) -o $@ libfuse/build/libfuse.a $(LDFLAGS) + mergerfs: build/mergerfs +tests: build/tests + changelog: ifeq ($(GIT_REPO),1) $(GIT2DEBCL) --name mergerfs > ChangeLog @@ -127,12 +145,16 @@ version: tools/update-version build/stamp: - $(MKDIR) -p build + $(MKDIR) -p build/.src build/.tests $(TOUCH) $@ -build/%.o: src/%.cpp +build/.src/%.o: src/%.cpp $(CXX) $(CXXFLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@ +build/.tests/%.o: tests/%.cpp + $(CXX) $(CXXFLAGS) $(TESTS_FLAGS) $(FUSE_FLAGS) $(MFS_FLAGS) $(CPPFLAGS) -c $< -o $@ + + .PHONY: clean clean: rpm-clean $(RM) -rf build diff --git a/src/branch.cpp b/src/branch.cpp index 34d79442..a693b8df 100644 --- a/src/branch.cpp +++ b/src/branch.cpp @@ -18,21 +18,8 @@ #include "branch.hpp" #include "ef.hpp" -#include "from_string.hpp" -#include "fs_glob.hpp" -#include "fs_realpathize.hpp" -#include "nonstd/optional.hpp" +#include "errno.hpp" #include "num.hpp" -#include "str.hpp" - -#include - -#include -#include - -using std::string; -using std::vector; -using nonstd::optional; Branch::Branch(const uint64_t &default_minfreespace_) @@ -108,354 +95,3 @@ Branch::ro_or_nc(void) const return ((mode == Branch::Mode::RO) || (mode == Branch::Mode::NC)); } - -namespace l -{ - static - void - split(const std::string &s_, - std::string *instr_, - std::string *values_) - { - uint64_t offset; - - offset = s_.find_first_of('/'); - *instr_ = s_.substr(0,offset); - if(offset != std::string::npos) - *values_ = s_.substr(offset); - } -} - -Branches::Branches(const uint64_t &default_minfreespace_) - : default_minfreespace(default_minfreespace_) -{ - pthread_rwlock_init(&lock,NULL); -} - -namespace l -{ - static - int - parse_mode(const string &str_, - Branch::Mode *mode_) - { - if(str_ == "RW") - *mode_ = Branch::Mode::RW; - ef(str_ == "RO") - *mode_ = Branch::Mode::RO; - ef(str_ == "NC") - *mode_ = Branch::Mode::NC; - else - return -EINVAL; - - return 0; - } - - static - int - parse_minfreespace(const string &str_, - optional *minfreespace_) - { - int rv; - uint64_t uint64; - - rv = str::from(str_,&uint64); - if(rv < 0) - return rv; - - *minfreespace_ = uint64; - - return 0; - } - - static - int - parse_branch(const string &str_, - string *glob_, - Branch::Mode *mode_, - optional *minfreespace_) - { - int rv; - string options; - vector v; - - str::rsplit1(str_,'=',&v); - switch(v.size()) - { - case 1: - *glob_ = v[0]; - *mode_ = Branch::Mode::RW; - break; - case 2: - *glob_ = v[0]; - options = v[1]; - v.clear(); - str::split(options,',',&v); - switch(v.size()) - { - case 2: - rv = l::parse_minfreespace(v[1],minfreespace_); - if(rv < 0) - return rv; - case 1: - rv = l::parse_mode(v[0],mode_); - if(rv < 0) - return rv; - break; - case 0: - return -EINVAL; - } - break; - default: - return -EINVAL; - } - - return 0; - } - - static - int - parse(const string &str_, - const uint64_t &default_minfreespace_, - BranchVec *branches_) - { - int rv; - string glob; - vector globbed; - optional minfreespace; - Branch branch(default_minfreespace_); - - rv = l::parse_branch(str_,&glob,&branch.mode,&minfreespace); - if(rv < 0) - return rv; - - if(minfreespace.has_value()) - branch.set_minfreespace(minfreespace.value()); - - fs::glob(glob,&globbed); - fs::realpathize(&globbed); - for(size_t i = 0; i < globbed.size(); i++) - { - branch.path = globbed[i]; - branches_->push_back(branch); - } - - return 0; - } - - static - int - set(const std::string &str_, - Branches *branches_) - { - int rv; - vector paths; - BranchVec tmp_branchvec; - - str::split(str_,':',&paths); - - for(size_t i = 0; i < paths.size(); i++) - { - rv = l::parse(paths[i],branches_->default_minfreespace,&tmp_branchvec); - if(rv < 0) - return rv; - } - - branches_->vec.clear(); - branches_->vec.insert(branches_->vec.end(), - tmp_branchvec.begin(), - tmp_branchvec.end()); - - return 0; - } - - static - int - add_begin(const std::string &str_, - Branches *branches_) - { - int rv; - vector paths; - BranchVec tmp_branchvec; - - str::split(str_,':',&paths); - - for(size_t i = 0; i < paths.size(); i++) - { - rv = l::parse(paths[i],branches_->default_minfreespace,&tmp_branchvec); - if(rv < 0) - return rv; - } - - branches_->vec.insert(branches_->vec.begin(), - tmp_branchvec.begin(), - tmp_branchvec.end()); - - return 0; - } - - static - int - add_end(const std::string &str_, - Branches *branches_) - { - int rv; - vector paths; - BranchVec tmp_branchvec; - - str::split(str_,':',&paths); - - for(size_t i = 0; i < paths.size(); i++) - { - rv = l::parse(paths[i],branches_->default_minfreespace,&tmp_branchvec); - if(rv < 0) - return rv; - } - - branches_->vec.insert(branches_->vec.end(), - tmp_branchvec.begin(), - tmp_branchvec.end()); - - return 0; - } - - static - int - erase_begin(BranchVec *branches_) - { - branches_->erase(branches_->begin()); - - return 0; - } - - static - int - erase_end(BranchVec *branches_) - { - branches_->pop_back(); - - return 0; - } - - static - int - erase_fnmatch(const std::string &str_, - Branches *branches_) - { - vector patterns; - - str::split(str_,':',&patterns); - - for(BranchVec::iterator i = branches_->vec.begin(); - i != branches_->vec.end();) - { - int match = FNM_NOMATCH; - - for(vector::const_iterator pi = patterns.begin(); - pi != patterns.end() && match != 0; - ++pi) - { - match = ::fnmatch(pi->c_str(),i->path.c_str(),0); - } - - i = ((match == 0) ? branches_->vec.erase(i) : (i+1)); - } - - return 0; - } -} - -int -Branches::from_string(const std::string &s_) -{ - rwlock::WriteGuard guard(lock); - - std::string instr; - std::string values; - - l::split(s_,&instr,&values); - - if(instr == "+") - return l::add_end(values,this); - if(instr == "+<") - return l::add_begin(values,this); - if(instr == "+>") - return l::add_end(values,this); - if(instr == "-") - return l::erase_fnmatch(values,this); - if(instr == "-<") - return l::erase_begin(&vec); - if(instr == "->") - return l::erase_end(&vec); - if(instr == "=") - return l::set(values,this); - if(instr.empty()) - return l::set(values,this); - - return -EINVAL; -} - -string -Branches::to_string(void) const -{ - rwlock::ReadGuard guard(lock); - - string tmp; - - for(size_t i = 0; i < vec.size(); i++) - { - const Branch &branch = vec[i]; - - tmp += branch.to_string(); - tmp += ':'; - } - - if(*tmp.rbegin() == ':') - tmp.erase(tmp.size() - 1); - - return tmp; -} - -void -Branches::to_paths(vector &vec_) const -{ - rwlock::ReadGuard guard(lock); - - for(size_t i = 0; i < vec.size(); i++) - { - const Branch &branch = vec[i]; - - vec_.push_back(branch.path); - } -} - -SrcMounts::SrcMounts(Branches &b_) - : _branches(b_) -{ - -} - -int -SrcMounts::from_string(const std::string &s_) -{ - return _branches.from_string(s_); -} - -std::string -SrcMounts::to_string(void) const -{ - rwlock::ReadGuard guard(_branches.lock); - - std::string rv; - - for(uint64_t i = 0; i < _branches.vec.size(); i++) - { - rv += _branches.vec[i].path; - rv += ':'; - } - - if(*rv.rbegin() == ':') - rv.erase(rv.size() - 1); - - return rv; -} diff --git a/src/branch.hpp b/src/branch.hpp index 421f0e3c..588aa79a 100644 --- a/src/branch.hpp +++ b/src/branch.hpp @@ -18,81 +18,50 @@ #pragma once -#include "rwlock.hpp" -#include "tofrom_string.hpp" #include "nonstd/optional.hpp" +#include "strvec.hpp" +#include "tofrom_string.hpp" +#include #include #include -#include -#include -class Branch : public ToFromString +class Branch final : public ToFromString { public: - Branch(const uint64_t &default_minfreespace); + typedef std::vector Vector; public: - int from_string(const std::string &str); - std::string to_string(void) const; + Branch(const uint64_t &default_minfreespace_); public: enum class Mode { - INVALID, - RO, - RW, - NC + INVALID, + RO, + RW, + NC }; -public: - Mode mode; - std::string path; - uint64_t minfreespace() const; - -public: - void set_minfreespace(const uint64_t minfreespace); - public: bool ro(void) const; bool nc(void) const; bool ro_or_nc(void) const; -private: - nonstd::optional _minfreespace; - const uint64_t *_default_minfreespace; -}; - -typedef std::vector BranchVec; - -class Branches : public ToFromString -{ -public: - Branches(const uint64_t &default_minfreespace_); - -public: - int from_string(const std::string &str); - std::string to_string(void) const; - -public: - void to_paths(std::vector &vec) const; - public: - mutable pthread_rwlock_t lock; - BranchVec vec; - const uint64_t &default_minfreespace; -}; + int from_string(const std::string &str) final; + std::string to_string(void) const final; -class SrcMounts : public ToFromString -{ public: - SrcMounts(Branches &b_); + uint64_t minfreespace() const; + void set_minfreespace(const uint64_t); public: - int from_string(const std::string &str); - std::string to_string(void) const; + Mode mode; + std::string path; private: - Branches &_branches; + nonstd::optional _minfreespace; + const uint64_t *_default_minfreespace; }; diff --git a/src/branches.cpp b/src/branches.cpp new file mode 100644 index 00000000..b8ca5ebb --- /dev/null +++ b/src/branches.cpp @@ -0,0 +1,429 @@ +/* + ISC License + + Copyright (c) 2021, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#include "branches.hpp" +#include "ef.hpp" +#include "errno.hpp" +#include "from_string.hpp" +#include "fs_glob.hpp" +#include "fs_realpathize.hpp" +#include "nonstd/optional.hpp" +#include "num.hpp" +#include "str.hpp" + +#include + +#include + +using std::string; +using std::vector; +using nonstd::optional; + + +Branches::Impl::Impl(const uint64_t &default_minfreespace_) + : _default_minfreespace(default_minfreespace_) +{ +} + +Branches::Impl& +Branches::Impl::operator=(Branches::Impl &rval_) +{ + auto this_base = dynamic_cast(this); + auto rval_base = dynamic_cast(&rval_); + + *this_base = *rval_base; + + return *this; +} + +Branches::Impl& +Branches::Impl::operator=(Branches::Impl &&rval_) +{ + auto this_base = dynamic_cast(this); + auto rval_base = dynamic_cast(&rval_); + + *this_base = std::move(*rval_base); + + return *this; +} + +const +uint64_t& +Branches::Impl::minfreespace(void) const +{ + return _default_minfreespace; +} + +namespace l +{ + static + void + split(const std::string &s_, + std::string *instr_, + std::string *values_) + { + uint64_t offset; + + offset = s_.find_first_of('/'); + *instr_ = s_.substr(0,offset); + if(offset != std::string::npos) + *values_ = s_.substr(offset); + } + + static + int + parse_mode(const string &str_, + Branch::Mode *mode_) + { + if(str_ == "RW") + *mode_ = Branch::Mode::RW; + ef(str_ == "RO") + *mode_ = Branch::Mode::RO; + ef(str_ == "NC") + *mode_ = Branch::Mode::NC; + else + return -EINVAL; + + return 0; + } + + static + int + parse_minfreespace(const string &str_, + optional *minfreespace_) + { + int rv; + uint64_t uint64; + + rv = str::from(str_,&uint64); + if(rv < 0) + return rv; + + *minfreespace_ = uint64; + + return 0; + } + + static + int + parse_branch(const string &str_, + string *glob_, + Branch::Mode *mode_, + optional *minfreespace_) + { + int rv; + string options; + vector v; + + str::rsplit1(str_,'=',&v); + switch(v.size()) + { + case 1: + *glob_ = v[0]; + *mode_ = Branch::Mode::RW; + break; + case 2: + *glob_ = v[0]; + options = v[1]; + v.clear(); + str::split(options,',',&v); + switch(v.size()) + { + case 2: + rv = l::parse_minfreespace(v[1],minfreespace_); + if(rv < 0) + return rv; + case 1: + rv = l::parse_mode(v[0],mode_); + if(rv < 0) + return rv; + break; + case 0: + return -EINVAL; + } + break; + default: + return -EINVAL; + } + + return 0; + } + + static + int + parse(const string &str_, + Branches::Impl *branches_) + { + int rv; + string glob; + StrVec paths; + optional minfreespace; + Branch branch(branches_->minfreespace()); + + rv = l::parse_branch(str_,&glob,&branch.mode,&minfreespace); + if(rv < 0) + return rv; + + if(minfreespace.has_value()) + branch.set_minfreespace(minfreespace.value()); + + fs::glob(glob,&paths); + fs::realpathize(&paths); + for(auto &path : paths) + { + branch.path = path; + branches_->push_back(branch); + } + + return 0; + } + + static + int + set(const std::string &str_, + Branches::Impl *branches_) + { + int rv; + StrVec paths; + Branches::Impl tmp_branches(branches_->minfreespace()); + + str::split(str_,':',&paths); + for(auto &path : paths) + { + rv = l::parse(path,&tmp_branches); + if(rv < 0) + return rv; + } + + *branches_ = std::move(tmp_branches); + + return 0; + } + + static + int + add_begin(const std::string &str_, + Branches::Impl *branches_) + { + int rv; + vector paths; + Branches::Impl tmp_branches(branches_->minfreespace()); + + str::split(str_,':',&paths); + for(auto &path : paths) + { + rv = l::parse(path,&tmp_branches); + if(rv < 0) + return rv; + } + + branches_->insert(branches_->begin(), + tmp_branches.begin(), + tmp_branches.end()); + + return 0; + } + + static + int + add_end(const std::string &str_, + Branches::Impl *branches_) + { + int rv; + StrVec paths; + Branches::Impl tmp_branches(branches_->minfreespace()); + + str::split(str_,':',&paths); + for(auto &path : paths) + { + rv = l::parse(path,&tmp_branches); + if(rv < 0) + return rv; + } + + branches_->insert(branches_->end(), + tmp_branches.begin(), + tmp_branches.end()); + + return 0; + } + + static + int + erase_begin(Branches::Impl *branches_) + { + branches_->erase(branches_->begin()); + + return 0; + } + + static + int + erase_end(Branches::Impl *branches_) + { + branches_->pop_back(); + + return 0; + } + + static + int + erase_fnmatch(const std::string &str_, + Branches::Impl *branches_) + { + StrVec patterns; + + str::split(str_,':',&patterns); + for(auto i = branches_->begin(); i != branches_->end();) + { + int match = FNM_NOMATCH; + + for(auto pi = patterns.cbegin(); pi != patterns.cend() && match != 0; ++pi) + { + match = ::fnmatch(pi->c_str(),i->path.c_str(),0); + } + + i = ((match == 0) ? branches_->erase(i) : (i+1)); + } + + return 0; + } +} + +int +Branches::Impl::from_string(const std::string &s_) +{ + std::string instr; + std::string values; + + l::split(s_,&instr,&values); + + if(instr == "+") + return l::add_end(values,this); + if(instr == "+<") + return l::add_begin(values,this); + if(instr == "+>") + return l::add_end(values,this); + if(instr == "-") + return l::erase_fnmatch(values,this); + if(instr == "-<") + return l::erase_begin(this); + if(instr == "->") + return l::erase_end(this); + if(instr == "=") + return l::set(values,this); + if(instr.empty()) + return l::set(values,this); + + return -EINVAL; +} + +std::string +Branches::Impl::to_string(void) const +{ + string tmp; + + if(empty()) + return tmp; + + for(auto &branch : *this) + { + tmp += branch.to_string(); + tmp += ':'; + } + + tmp.pop_back(); + + return tmp; +} + +void +Branches::Impl::to_paths(StrVec &paths_) const +{ + for(auto &branch : *this) + { + paths_.push_back(branch.path); + } +} + +int +Branches::from_string(const std::string &str_) +{ + int rv; + Branches::Ptr impl; + Branches::Ptr new_impl; + + { + std::lock_guard lock_guard(_mutex); + impl = _impl; + } + + new_impl = std::make_shared(impl->minfreespace()); + *new_impl = *impl; + + rv = new_impl->from_string(str_); + if(rv < 0) + return rv; + + { + std::lock_guard lock_guard(_mutex); + _impl = new_impl; + } + + return 0; +} + +string +Branches::to_string(void) const +{ + std::lock_guard lock_guard(_mutex); + + return _impl->to_string(); +} + +SrcMounts::SrcMounts(Branches &b_) + : _branches(b_) +{ + +} + +int +SrcMounts::from_string(const std::string &s_) +{ + return _branches.from_string(s_); +} + +std::string +SrcMounts::to_string(void) const +{ + std::string rv; + Branches::CPtr branches = _branches; + + if(branches->empty()) + return rv; + + for(const auto &branch : *branches) + { + rv += branch.path; + rv += ':'; + } + + rv.pop_back(); + + return rv; +} diff --git a/src/branches.hpp b/src/branches.hpp new file mode 100644 index 00000000..28731847 --- /dev/null +++ b/src/branches.hpp @@ -0,0 +1,94 @@ +/* + ISC License + + Copyright (c) 2021, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "branch.hpp" +#include "nonstd/optional.hpp" +#include "strvec.hpp" +#include "tofrom_string.hpp" + +#include +#include +#include +#include +#include + + +class Branches final : public ToFromString +{ +public: + class Impl final : public ToFromString, public Branch::Vector + { + public: + typedef std::shared_ptr Ptr; + typedef std::shared_ptr CPtr; + + public: + Impl(const uint64_t &default_minfreespace_); + + public: + int from_string(const std::string &str) final; + std::string to_string(void) const final; + + public: + const uint64_t& minfreespace(void) const; + void to_paths(StrVec &strvec) const; + + public: + Impl& operator=(Impl &impl_); + Impl& operator=(Impl &&impl_); + + private: + const uint64_t &_default_minfreespace; + }; + +public: + typedef Branches::Impl::Ptr Ptr; + typedef Branches::Impl::CPtr CPtr; + +public: + Branches(const uint64_t &default_minfreespace_) + : _impl(std::make_shared(default_minfreespace_)) + {} + +public: + int from_string(const std::string &str) final; + std::string to_string(void) const final; + +public: + operator CPtr() const { std::lock_guard lg(_mutex); return _impl; } + CPtr operator->() const { std::lock_guard lg(_mutex); return _impl; } + +private: + mutable std::mutex _mutex; + Ptr _impl; +}; + +class SrcMounts : public ToFromString +{ +public: + SrcMounts(Branches &b_); + +public: + int from_string(const std::string &str) final; + std::string to_string(void) const final; + +private: + Branches &_branches; +}; diff --git a/src/category.cpp b/src/category.cpp new file mode 100644 index 00000000..e1fd7f2a --- /dev/null +++ b/src/category.cpp @@ -0,0 +1,49 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#include "category.hpp" +#include "errno.hpp" +#include "str.hpp" + +#include + +int +Category::Base::from_string(const std::string &s_) +{ + int rv; + + for(auto func : funcs) + { + rv = func->from_string(s_); + if(rv < 0) + return rv; + } + + return 0; +} + +std::string +Category::Base::to_string(void) const +{ + std::set rv; + + for(auto func : funcs) + rv.insert(func->to_string()); + + return str::join(rv,','); +} diff --git a/src/category.hpp b/src/category.hpp index 08a7c00a..ff3b87f2 100644 --- a/src/category.hpp +++ b/src/category.hpp @@ -1,5 +1,5 @@ /* - Copyright (c) 2016, Antonio SJ Musumeci + Copyright (c) 2020, Antonio SJ Musumeci Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above @@ -16,9 +16,92 @@ #pragma once -enum class Category +#include "tofrom_string.hpp" +#include "funcs.hpp" +#include "func.hpp" + +#include + +namespace Category +{ + class Base : public ToFromString + { + public: + int from_string(const std::string &s) final; + std::string to_string() const final; + + protected: + std::vector funcs; + }; + + class Action final : public Base + { + private: + Action(); + + public: + Action(Funcs &funcs_) + { + funcs.push_back(&funcs_.chmod); + funcs.push_back(&funcs_.chown); + funcs.push_back(&funcs_.link); + funcs.push_back(&funcs_.removexattr); + funcs.push_back(&funcs_.rename); + funcs.push_back(&funcs_.rmdir); + funcs.push_back(&funcs_.setxattr); + funcs.push_back(&funcs_.truncate); + funcs.push_back(&funcs_.unlink); + funcs.push_back(&funcs_.utimens); + } + }; + + class Create final : public Base { - ACTION, - CREATE, - SEARCH + private: + Create(); + + public: + Create(Funcs &funcs_) + { + funcs.push_back(&funcs_.create); + funcs.push_back(&funcs_.mkdir); + funcs.push_back(&funcs_.mknod); + funcs.push_back(&funcs_.symlink); + } }; + + class Search final : public Base + { + private: + Search(); + + public: + Search(Funcs &funcs_) + { + funcs.push_back(&funcs_.access); + funcs.push_back(&funcs_.getattr); + funcs.push_back(&funcs_.getxattr); + funcs.push_back(&funcs_.listxattr); + funcs.push_back(&funcs_.open); + funcs.push_back(&funcs_.readlink); + } + }; +} + +class Categories final +{ +private: + Categories(); + +public: + Categories(Funcs &funcs_) + : action(funcs_), + create(funcs_), + search(funcs_) + {} + +public: + Category::Action action; + Category::Create create; + Category::Search search; +}; diff --git a/src/config.cpp b/src/config.cpp index 6c8e1064..aadb2ae6 100644 --- a/src/config.cpp +++ b/src/config.cpp @@ -20,14 +20,18 @@ #include "from_string.hpp" #include "num.hpp" #include "rwlock.hpp" +#include "str.hpp" #include "to_string.hpp" #include "version.hpp" #include -#include +#include +#include #include +#include -#include +#include +#include #include #include @@ -37,6 +41,11 @@ using std::string; #define IFERT(S) if(S == s_) return true +const std::string CONTROLFILE = "/.mergerfs"; + +Config Config::_singleton; + + namespace l { static @@ -60,49 +69,44 @@ namespace l } Config::Config() - : - open_cache(), - - controlfile("/.mergerfs"), - - async_read(true), - auto_cache(false), - branches(minfreespace), - cache_attr(1), - cache_entry(1), - cache_files(CacheFiles::ENUM::LIBFUSE), - cache_negative_entry(0), - cache_readdir(false), - cache_statfs(0), - cache_symlinks(false), - category(func), - direct_io(false), - dropcacheonclose(false), - fsname(), - func(), - fuse_msg_size(FUSE_MAX_MAX_PAGES), - ignorepponrename(false), - inodecalc("hybrid-hash"), - link_cow(false), - minfreespace(MINFREESPACE_DEFAULT), - mount(), - moveonenospc(false), - nfsopenhack(NFSOpenHack::ENUM::OFF), - nullrw(false), - pid(::getpid()), - posix_acl(false), - readdir(ReadDir::ENUM::POSIX), - readdirplus(false), - security_capability(true), - srcmounts(branches), - statfs(StatFS::ENUM::BASE), - statfs_ignore(StatFSIgnore::ENUM::NONE), - symlinkify(false), - symlinkify_timeout(3600), - threads(0), - version(MERGERFS_VERSION), - writeback_cache(false), - xattr(XAttr::ENUM::PASSTHROUGH) + : async_read(true), + auto_cache(false), + branches(minfreespace), + cache_attr(1), + cache_entry(1), + cache_files(CacheFiles::ENUM::LIBFUSE), + cache_negative_entry(0), + cache_readdir(false), + cache_statfs(0), + cache_symlinks(false), + category(func), + direct_io(false), + dropcacheonclose(false), + fsname(), + func(), + fuse_msg_size(FUSE_MAX_MAX_PAGES), + ignorepponrename(false), + inodecalc("hybrid-hash"), + link_cow(false), + minfreespace(MINFREESPACE_DEFAULT), + mount(), + moveonenospc(false), + nfsopenhack(NFSOpenHack::ENUM::OFF), + nullrw(false), + pid(::getpid()), + posix_acl(false), + readdir(ReadDir::ENUM::POSIX), + readdirplus(false), + security_capability(true), + srcmounts(branches), + statfs(StatFS::ENUM::BASE), + statfs_ignore(StatFSIgnore::ENUM::NONE), + symlinkify(false), + symlinkify_timeout(3600), + threads(0), + version(MERGERFS_VERSION), + writeback_cache(false), + xattr(XAttr::ENUM::PASSTHROUGH) { _map["async_read"] = &async_read; _map["auto_cache"] = &auto_cache; @@ -166,17 +170,22 @@ Config::Config() _map["xattr"] = &xattr; } -const Config& -Config::ro(void) +Config::operator=(const Config &cfg_) { - return *((Config*)fuse_get_context()->private_data); -} + int rv; + std::string val; -Config& -Config::rw(void) -{ - return *((Config*)fuse_get_context()->private_data); + for(auto &kv : _map) + { + rv = cfg_.get(kv.first,&val); + if(rv) + continue; + + kv.second->from_string(val); + } + + return *this; } bool @@ -254,11 +263,80 @@ Config::set(const std::string &key_, const std::string &value_) { if(l::readonly(key_)) - return -EINVAL; + return -EROFS; return set_raw(key_,value_); } +int +Config::set(const std::string &kv_) +{ + std::string key; + std::string val; + + str::splitkv(kv_,'=',&key,&val); + key = str::trim(key); + val = str::trim(val); + + return set(key,val); +} + +int +Config::from_stream(std::istream &istrm_, + ErrVec *errs_) +{ + int rv; + std::string line; + std::string key; + std::string val; + Config newcfg; + + newcfg = *this; + + while(std::getline(istrm_,line,'\n')) + { + line = str::trim(line); + if(!line.empty() && (line[0] == '#')) + continue; + + str::splitkv(line,'=',&key,&val); + key = str::trim(key); + val = str::trim(val); + + rv = newcfg.set(key,val); + if(rv < 0) + errs_->push_back({rv,key}); + } + + if(!errs_->empty()) + return -EINVAL; + + *this = newcfg; + + return 0; +} + +int +Config::from_file(const std::string &filepath_, + ErrVec *errs_) +{ + int rv; + std::ifstream ifstrm; + + ifstrm.open(filepath_); + if(!ifstrm.good()) + { + errs_->push_back({-errno,filepath_}); + return -errno; + } + + rv = from_stream(ifstrm,errs_); + + ifstrm.close(); + + return rv; +} + std::ostream& operator<<(std::ostream &os_, const Config &c_) @@ -268,7 +346,47 @@ operator<<(std::ostream &os_, for(i = c_._map.begin(), ei = c_._map.end(); i != ei; ++i) { - os_ << i->first << '=' << i->second << '\n'; + os_ << i->first << '=' << i->second->to_string() << std::endl; + } + + return os_; +} + + +static +std::string +err2str(const int err_) +{ + switch(err_) + { + case 0: + return std::string(); + case -EINVAL: + return "invalid value"; + case -ENOATTR: + return "unknown option"; + case -EROFS: + return "read-only option"; + default: + return strerror(-err_); + } + + return std::string(); +} + +std::ostream& +operator<<(std::ostream &os_, + const Config::ErrVec &ev_) +{ + std::string errstr; + + for(auto &err : ev_) + { + os_ << "* ERROR: "; + errstr = err2str(err.err); + if(!errstr.empty()) + os_ << errstr << " - "; + os_ << err.str << std::endl; } return os_; diff --git a/src/config.hpp b/src/config.hpp index c66f3815..6bd1d02b 100644 --- a/src/config.hpp +++ b/src/config.hpp @@ -16,7 +16,8 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" +#include "category.hpp" #include "config_cachefiles.hpp" #include "config_inodecalc.hpp" #include "config_moveonenospc.hpp" @@ -27,18 +28,20 @@ #include "config_xattr.hpp" #include "enum.hpp" #include "errno.hpp" -#include "func_category.hpp" #include "funcs.hpp" #include "policy.hpp" -#include "policy_cache.hpp" +#include "rwlock.hpp" #include "tofrom_wrapper.hpp" -#include +#include "fuse.h" +#include +#include +#include +#include #include #include -#include #include typedef ToFromWrapper ConfigBOOL; @@ -47,16 +50,50 @@ typedef ToFromWrapper ConfigINT; typedef ToFromWrapper ConfigSTR; typedef std::map Str2TFStrMap; +extern const std::string CONTROLFILE; + class Config { public: - Config(); + struct Err + { + int err; + std::string str; + }; + + typedef std::vector ErrVec; + +public: + class Read + { + public: + Read(); + + public: + inline const Config* operator->() const; + + private: + const Config &_cfg; + }; public: - mutable PolicyCache open_cache; + class Write + { + public: + Write(); + + public: + Config* operator->(); + + private: + Config &_cfg; + }; public: - const std::string controlfile; + Config(); + +public: + Config& operator=(const Config&); public: ConfigBOOL async_read; @@ -69,7 +106,7 @@ public: ConfigBOOL cache_readdir; ConfigUINT64 cache_statfs; ConfigBOOL cache_symlinks; - FuncCategories category; + Categories category; ConfigBOOL direct_io; ConfigBOOL dropcacheonclose; ConfigSTR fsname; @@ -112,11 +149,50 @@ public: int get(const std::string &key, std::string *val) const; int set_raw(const std::string &key, const std::string &val); int set(const std::string &key, const std::string &val); + int set(const std::string &kv); public: - static const Config &ro(void); - static Config &rw(void); + int from_stream(std::istream &istrm, ErrVec *errs); + int from_file(const std::string &filepath, ErrVec *errs); private: Str2TFStrMap _map; + +private: + static Config _singleton; + +public: + friend class Read; + friend class Write; }; + +std::ostream& operator<<(std::ostream &s,const Config::ErrVec &ev); + +inline +Config::Read::Read() + : _cfg(Config::_singleton) +{ + +} + +inline +const +Config* +Config::Read::operator->() const +{ + return &_cfg; +} + +inline +Config::Write::Write() + : _cfg(Config::_singleton) +{ + +} + +inline +Config* +Config::Write::operator->() +{ + return &_cfg; +} diff --git a/src/config_cachefiles.hpp b/src/config_cachefiles.hpp index c2df5d7e..b6c9a2dc 100644 --- a/src/config_cachefiles.hpp +++ b/src/config_cachefiles.hpp @@ -20,6 +20,7 @@ #include "enum.hpp" + enum class CacheFilesEnum { LIBFUSE, diff --git a/src/config_inodecalc.cpp b/src/config_inodecalc.cpp index b471004d..aa21e048 100644 --- a/src/config_inodecalc.cpp +++ b/src/config_inodecalc.cpp @@ -19,6 +19,7 @@ #include "config_inodecalc.hpp" #include "fs_inode.hpp" + InodeCalc::InodeCalc(const std::string &s_) { fs::inode::set_algo(s_); diff --git a/src/config_inodecalc.hpp b/src/config_inodecalc.hpp index b87b7fc5..8a882593 100644 --- a/src/config_inodecalc.hpp +++ b/src/config_inodecalc.hpp @@ -20,12 +20,13 @@ #include "tofrom_string.hpp" + class InodeCalc : public ToFromString { public: InodeCalc(const std::string &); public: - std::string to_string(void) const; - int from_string(const std::string &); + std::string to_string(void) const final; + int from_string(const std::string &) final; }; diff --git a/src/config_moveonenospc.cpp b/src/config_moveonenospc.cpp index a1975459..17950cc8 100644 --- a/src/config_moveonenospc.cpp +++ b/src/config_moveonenospc.cpp @@ -21,12 +21,13 @@ #include "errno.hpp" #include "from_string.hpp" + int MoveOnENOSPC::from_string(const std::string &s_) { int rv; std::string s; - const Policy *tmp; + Policy::CreateImpl *tmp; rv = str::from(s_,&enabled); if((rv == 0) && (enabled == true)) @@ -36,8 +37,8 @@ MoveOnENOSPC::from_string(const std::string &s_) else return 0; - tmp = &Policy::find(s); - if(tmp == Policy::invalid) + tmp = Policies::Create::find(s); + if(tmp == NULL) return -EINVAL; policy = tmp; @@ -50,6 +51,6 @@ std::string MoveOnENOSPC::to_string(void) const { if(enabled) - return policy->to_string(); + return policy.name(); return "false"; } diff --git a/src/config_moveonenospc.hpp b/src/config_moveonenospc.hpp index a7aa4fe1..2aac71fb 100644 --- a/src/config_moveonenospc.hpp +++ b/src/config_moveonenospc.hpp @@ -19,26 +19,26 @@ #pragma once #include "policy.hpp" +#include "policies.hpp" #include "tofrom_string.hpp" #include + class MoveOnENOSPC : public ToFromString { public: MoveOnENOSPC(const bool enabled_) - : enabled(enabled_) + : enabled(enabled_), + policy(&Policies::Create::mfs) { - policy = (enabled ? - &Policy::mfs : - &Policy::invalid); } public: - int from_string(const std::string &s); - std::string to_string() const; + int from_string(const std::string &s) final; + std::string to_string() const final; public: bool enabled; - const Policy *policy; + Policy::Create policy; }; diff --git a/src/config_nfsopenhack.cpp b/src/config_nfsopenhack.cpp index 81c9b169..55cad062 100644 --- a/src/config_nfsopenhack.cpp +++ b/src/config_nfsopenhack.cpp @@ -20,6 +20,7 @@ #include "ef.hpp" #include "errno.hpp" + template<> int NFSOpenHack::from_string(const std::string &s_) diff --git a/src/config_nfsopenhack.hpp b/src/config_nfsopenhack.hpp index e6a3e797..0e54b482 100644 --- a/src/config_nfsopenhack.hpp +++ b/src/config_nfsopenhack.hpp @@ -20,6 +20,7 @@ #include "enum.hpp" + enum class NFSOpenHackEnum { OFF, diff --git a/src/config_readdir.cpp b/src/config_readdir.cpp index b0dd07bd..f03b2a77 100644 --- a/src/config_readdir.cpp +++ b/src/config_readdir.cpp @@ -20,6 +20,7 @@ #include "ef.hpp" #include "errno.hpp" + template<> int ReadDir::from_string(const std::string &s_) diff --git a/src/config_readdir.hpp b/src/config_readdir.hpp index fe041d87..96767f20 100644 --- a/src/config_readdir.hpp +++ b/src/config_readdir.hpp @@ -20,6 +20,7 @@ #include "enum.hpp" + enum class ReadDirEnum { POSIX, diff --git a/src/config_statfs.cpp b/src/config_statfs.cpp index 07642786..50910367 100644 --- a/src/config_statfs.cpp +++ b/src/config_statfs.cpp @@ -20,6 +20,7 @@ #include "ef.hpp" #include "errno.hpp" + template<> std::string StatFS::to_string() const diff --git a/src/config_statfs.hpp b/src/config_statfs.hpp index f217371b..0d408aeb 100644 --- a/src/config_statfs.hpp +++ b/src/config_statfs.hpp @@ -20,6 +20,7 @@ #include "enum.hpp" + enum class StatFSEnum { BASE, diff --git a/src/config_statfsignore.cpp b/src/config_statfsignore.cpp index e3355053..14b5ded3 100644 --- a/src/config_statfsignore.cpp +++ b/src/config_statfsignore.cpp @@ -20,6 +20,7 @@ #include "ef.hpp" #include "errno.hpp" + template<> std::string StatFSIgnore::to_string() const diff --git a/src/config_xattr.cpp b/src/config_xattr.cpp index 8b243c0f..bb626f22 100644 --- a/src/config_xattr.cpp +++ b/src/config_xattr.cpp @@ -20,6 +20,7 @@ #include "ef.hpp" #include "errno.hpp" + template<> std::string XAttr::to_string() const diff --git a/src/config_xattr.hpp b/src/config_xattr.hpp index c9eac263..34329fa4 100644 --- a/src/config_xattr.hpp +++ b/src/config_xattr.hpp @@ -21,6 +21,7 @@ #include "enum.hpp" #include "errno.hpp" + enum class XAttrEnum { PASSTHROUGH = 0, diff --git a/src/dirinfo.hpp b/src/dirinfo.hpp index 37ee6d3b..7048fbf4 100644 --- a/src/dirinfo.hpp +++ b/src/dirinfo.hpp @@ -20,6 +20,7 @@ #include + class DirInfo : public FH { public: diff --git a/src/endian.hpp b/src/endian.hpp index d294dba8..d5454994 100644 --- a/src/endian.hpp +++ b/src/endian.hpp @@ -18,6 +18,7 @@ #pragma once + namespace endian { static diff --git a/src/enum.hpp b/src/enum.hpp index 6f0dd35a..848ffca8 100644 --- a/src/enum.hpp +++ b/src/enum.hpp @@ -22,6 +22,7 @@ #include + template class Enum : public ToFromString { diff --git a/src/fh.hpp b/src/fh.hpp index 2060e2a5..4187b7f5 100644 --- a/src/fh.hpp +++ b/src/fh.hpp @@ -18,6 +18,7 @@ #include + class FH { public: diff --git a/src/fileinfo.hpp b/src/fileinfo.hpp index a3ca65b1..4d4948e4 100644 --- a/src/fileinfo.hpp +++ b/src/fileinfo.hpp @@ -20,6 +20,7 @@ #include + class FileInfo : public FH { public: diff --git a/src/fixed_mem_pool.hpp b/src/fixed_mem_pool.hpp index e81f1343..2f012b17 100644 --- a/src/fixed_mem_pool.hpp +++ b/src/fixed_mem_pool.hpp @@ -18,8 +18,10 @@ #pragma once +#include + #include -#include + typedef struct fixed_mem_pool_t fixed_mem_pool_t; struct fixed_mem_pool_t diff --git a/src/from_string.cpp b/src/from_string.cpp index 796f9256..638c8772 100644 --- a/src/from_string.cpp +++ b/src/from_string.cpp @@ -17,13 +17,14 @@ */ #include "ef.hpp" +#include "errno.hpp" +#include #include -#include -#include #include + namespace str { int @@ -50,7 +51,17 @@ namespace str from(const std::string &value_, int *int_) { - *int_ = ::strtol(value_.c_str(),NULL,10); + int tmp; + char *endptr; + + errno = 0; + tmp = ::strtol(value_.c_str(),&endptr,10); + if(errno != 0) + return -EINVAL; + if(endptr == value_.c_str()) + return -EINVAL; + + *int_ = tmp; return 0; } diff --git a/src/from_string.hpp b/src/from_string.hpp index 3c06884c..018bc2b7 100644 --- a/src/from_string.hpp +++ b/src/from_string.hpp @@ -18,9 +18,9 @@ #pragma once +#include #include -#include namespace str { diff --git a/src/fs_acl.cpp b/src/fs_acl.cpp index 148f21dc..3737bc16 100644 --- a/src/fs_acl.cpp +++ b/src/fs_acl.cpp @@ -23,6 +23,7 @@ const char POSIX_ACL_DEFAULT_XATTR[] = "system.posix_acl_default"; + namespace fs { namespace acl diff --git a/src/fs_acl.hpp b/src/fs_acl.hpp index f3b16e43..92fef84b 100644 --- a/src/fs_acl.hpp +++ b/src/fs_acl.hpp @@ -20,6 +20,7 @@ #include + namespace fs { namespace acl diff --git a/src/fs_attr.hpp b/src/fs_attr.hpp index c9bee84e..96d9944b 100644 --- a/src/fs_attr.hpp +++ b/src/fs_attr.hpp @@ -18,6 +18,7 @@ #include + namespace fs { namespace attr diff --git a/src/fs_attr_linux.icpp b/src/fs_attr_linux.icpp index 8d346190..abae1e5e 100644 --- a/src/fs_attr_linux.icpp +++ b/src/fs_attr_linux.icpp @@ -26,6 +26,7 @@ using std::string; + namespace fs { namespace attr diff --git a/src/fs_attr_unsupported.icpp b/src/fs_attr_unsupported.icpp index 852038fc..4a5c37bb 100644 --- a/src/fs_attr_unsupported.icpp +++ b/src/fs_attr_unsupported.icpp @@ -18,6 +18,7 @@ #include + namespace fs { namespace attr diff --git a/src/fs_clonefile.cpp b/src/fs_clonefile.cpp index 6dabde05..69d94c04 100644 --- a/src/fs_clonefile.cpp +++ b/src/fs_clonefile.cpp @@ -22,6 +22,7 @@ #include "fs_fchmod.hpp" #include "fs_futimens.hpp" + namespace l { static diff --git a/src/fs_clonefile.hpp b/src/fs_clonefile.hpp index a9835e57..67eeca33 100644 --- a/src/fs_clonefile.hpp +++ b/src/fs_clonefile.hpp @@ -16,6 +16,7 @@ #pragma once + namespace fs { int diff --git a/src/fs_clonepath.cpp b/src/fs_clonepath.cpp index 9e42946a..4fec3ae7 100644 --- a/src/fs_clonepath.cpp +++ b/src/fs_clonepath.cpp @@ -14,8 +14,6 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include - #include "errno.h" #include "fs_attr.hpp" #include "fs_clonepath.hpp" @@ -27,8 +25,11 @@ #include "fs_xattr.hpp" #include "ugid.hpp" +#include + using std::string; + namespace l { static diff --git a/src/fs_clonepath.hpp b/src/fs_clonepath.hpp index 34904d0c..8ad68c97 100644 --- a/src/fs_clonepath.hpp +++ b/src/fs_clonepath.hpp @@ -18,6 +18,7 @@ #include + namespace fs { int clonepath(const std::string &from, diff --git a/src/fs_close.hpp b/src/fs_close.hpp index 236d290d..646482fa 100644 --- a/src/fs_close.hpp +++ b/src/fs_close.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_closedir.hpp b/src/fs_closedir.hpp index 03ae8785..f7fc081f 100644 --- a/src/fs_closedir.hpp +++ b/src/fs_closedir.hpp @@ -21,6 +21,7 @@ #include #include + namespace fs { static diff --git a/src/fs_copy_file_range.hpp b/src/fs_copy_file_range.hpp index aab5fc9e..5ffdc656 100644 --- a/src/fs_copy_file_range.hpp +++ b/src/fs_copy_file_range.hpp @@ -18,10 +18,12 @@ #pragma once +#include + #include -#include #include + namespace fs { int64_t diff --git a/src/fs_copy_file_range_linux.icpp b/src/fs_copy_file_range_linux.icpp index 7df4f845..79103443 100644 --- a/src/fs_copy_file_range_linux.icpp +++ b/src/fs_copy_file_range_linux.icpp @@ -22,13 +22,15 @@ #include "errno.hpp" +#include + #include -#include #include #include #include #include + namespace l { static diff --git a/src/fs_copy_file_range_unsupported.icpp b/src/fs_copy_file_range_unsupported.icpp index c0e87fc8..2e87ecb7 100644 --- a/src/fs_copy_file_range_unsupported.icpp +++ b/src/fs_copy_file_range_unsupported.icpp @@ -18,10 +18,12 @@ #include "errno.h" +#include + #include -#include #include + namespace fs { ssize_t diff --git a/src/fs_copydata.cpp b/src/fs_copydata.cpp index 979b31b7..fee8821a 100644 --- a/src/fs_copydata.cpp +++ b/src/fs_copydata.cpp @@ -24,6 +24,7 @@ #include + namespace fs { int diff --git a/src/fs_copydata.hpp b/src/fs_copydata.hpp index 48e0784d..4347bcb6 100644 --- a/src/fs_copydata.hpp +++ b/src/fs_copydata.hpp @@ -20,6 +20,7 @@ #include + namespace fs { int diff --git a/src/fs_copydata_copy_file_range.cpp b/src/fs_copydata_copy_file_range.cpp index f8e7c388..5722750b 100644 --- a/src/fs_copydata_copy_file_range.cpp +++ b/src/fs_copydata_copy_file_range.cpp @@ -18,7 +18,8 @@ #include "fs_copy_file_range.hpp" #include "fs_fstat.hpp" -#include +#include + namespace l { diff --git a/src/fs_copydata_copy_file_range.hpp b/src/fs_copydata_copy_file_range.hpp index 6b0f7567..b22f53ec 100644 --- a/src/fs_copydata_copy_file_range.hpp +++ b/src/fs_copydata_copy_file_range.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include + namespace fs { diff --git a/src/fs_copydata_readwrite.cpp b/src/fs_copydata_readwrite.cpp index c119585d..e2c48795 100644 --- a/src/fs_copydata_readwrite.cpp +++ b/src/fs_copydata_readwrite.cpp @@ -24,6 +24,7 @@ using std::vector; + namespace l { static diff --git a/src/fs_copydata_readwrite.hpp b/src/fs_copydata_readwrite.hpp index 9e900d03..43271165 100644 --- a/src/fs_copydata_readwrite.hpp +++ b/src/fs_copydata_readwrite.hpp @@ -16,6 +16,7 @@ #pragma once + namespace fs { int diff --git a/src/fs_cow.cpp b/src/fs_cow.cpp index 63379fe5..37855329 100644 --- a/src/fs_cow.cpp +++ b/src/fs_cow.cpp @@ -34,6 +34,7 @@ using std::string; + namespace l { static diff --git a/src/fs_cow.hpp b/src/fs_cow.hpp index 5cb9353d..7403e5d9 100644 --- a/src/fs_cow.hpp +++ b/src/fs_cow.hpp @@ -21,6 +21,7 @@ #include #include + namespace fs { namespace cow diff --git a/src/fs_devid.hpp b/src/fs_devid.hpp index 2f9bde05..d75bcf3e 100644 --- a/src/fs_devid.hpp +++ b/src/fs_devid.hpp @@ -20,6 +20,7 @@ #include "fs_fstat.hpp" + namespace fs { static diff --git a/src/fs_dirfd.hpp b/src/fs_dirfd.hpp index d4afa76e..767d6685 100644 --- a/src/fs_dirfd.hpp +++ b/src/fs_dirfd.hpp @@ -21,6 +21,7 @@ #include #include + namespace fs { static diff --git a/src/fs_dup.hpp b/src/fs_dup.hpp index fc1dd106..fdb82c66 100644 --- a/src/fs_dup.hpp +++ b/src/fs_dup.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_eaccess.hpp b/src/fs_eaccess.hpp index f661566a..03f55287 100644 --- a/src/fs_eaccess.hpp +++ b/src/fs_eaccess.hpp @@ -20,6 +20,7 @@ #include "fs_faccessat.hpp" + namespace fs { static diff --git a/src/fs_exists.hpp b/src/fs_exists.hpp index 808bc8b1..2abdd646 100644 --- a/src/fs_exists.hpp +++ b/src/fs_exists.hpp @@ -23,6 +23,7 @@ #include + namespace fs { static diff --git a/src/fs_faccessat.hpp b/src/fs_faccessat.hpp index 02729b87..80329f47 100644 --- a/src/fs_faccessat.hpp +++ b/src/fs_faccessat.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_fadvise.hpp b/src/fs_fadvise.hpp index 96fa8360..e1126228 100644 --- a/src/fs_fadvise.hpp +++ b/src/fs_fadvise.hpp @@ -18,6 +18,7 @@ #include + namespace fs { int diff --git a/src/fs_fadvise_posix.icpp b/src/fs_fadvise_posix.icpp index a51e5de5..15bc8601 100644 --- a/src/fs_fadvise_posix.icpp +++ b/src/fs_fadvise_posix.icpp @@ -16,6 +16,7 @@ #include + namespace fs { static diff --git a/src/fs_fadvise_unsupported.icpp b/src/fs_fadvise_unsupported.icpp index b25aebcb..40617b01 100644 --- a/src/fs_fadvise_unsupported.icpp +++ b/src/fs_fadvise_unsupported.icpp @@ -16,6 +16,7 @@ #include "errno.hpp" + namespace fs { static diff --git a/src/fs_fallocate.hpp b/src/fs_fallocate.hpp index d6691472..74d33feb 100644 --- a/src/fs_fallocate.hpp +++ b/src/fs_fallocate.hpp @@ -18,6 +18,7 @@ #include + namespace fs { int diff --git a/src/fs_fallocate_linux.icpp b/src/fs_fallocate_linux.icpp index ae274eeb..f3eb2821 100644 --- a/src/fs_fallocate_linux.icpp +++ b/src/fs_fallocate_linux.icpp @@ -16,6 +16,7 @@ #include + namespace fs { int diff --git a/src/fs_fallocate_osx.icpp b/src/fs_fallocate_osx.icpp index d98c2795..746d7fb4 100644 --- a/src/fs_fallocate_osx.icpp +++ b/src/fs_fallocate_osx.icpp @@ -14,10 +14,11 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include "errno.hpp" + #include #include -#include "errno.hpp" namespace l { diff --git a/src/fs_fallocate_posix.icpp b/src/fs_fallocate_posix.icpp index 17f99e95..bbd2678c 100644 --- a/src/fs_fallocate_posix.icpp +++ b/src/fs_fallocate_posix.icpp @@ -18,6 +18,7 @@ #include + namespace fs { int diff --git a/src/fs_fallocate_unsupported.icpp b/src/fs_fallocate_unsupported.icpp index 9c913e84..c4211986 100644 --- a/src/fs_fallocate_unsupported.icpp +++ b/src/fs_fallocate_unsupported.icpp @@ -16,6 +16,7 @@ #include "errno.hpp" + namespace fs { int diff --git a/src/fs_fchmod.hpp b/src/fs_fchmod.hpp index 5300d43d..5d8c8195 100644 --- a/src/fs_fchmod.hpp +++ b/src/fs_fchmod.hpp @@ -24,6 +24,7 @@ #define MODE_BITS (S_ISUID|S_ISGID|S_ISVTX|S_IRWXU|S_IRWXG|S_IRWXO) + namespace fs { static diff --git a/src/fs_fchmodat.hpp b/src/fs_fchmodat.hpp index 209efbc6..e6e5ec9d 100644 --- a/src/fs_fchmodat.hpp +++ b/src/fs_fchmodat.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_fchown.hpp b/src/fs_fchown.hpp index 1b879e07..f8d92561 100644 --- a/src/fs_fchown.hpp +++ b/src/fs_fchown.hpp @@ -25,6 +25,7 @@ #include #include + namespace fs { static diff --git a/src/fs_fdatasync.hpp b/src/fs_fdatasync.hpp index 7b7fa72f..ac6183c1 100644 --- a/src/fs_fdatasync.hpp +++ b/src/fs_fdatasync.hpp @@ -26,6 +26,7 @@ #include + namespace fs { static diff --git a/src/fs_fgetxattr.hpp b/src/fs_fgetxattr.hpp index 57541eb0..7530d5d6 100644 --- a/src/fs_fgetxattr.hpp +++ b/src/fs_fgetxattr.hpp @@ -25,6 +25,7 @@ #include + namespace fs { static diff --git a/src/fs_ficlone.hpp b/src/fs_ficlone.hpp index 6900a82b..fd046c64 100644 --- a/src/fs_ficlone.hpp +++ b/src/fs_ficlone.hpp @@ -18,6 +18,7 @@ #pragma once + namespace fs { int diff --git a/src/fs_ficlone_linux.icpp b/src/fs_ficlone_linux.icpp index a36196fd..f0ce01bd 100644 --- a/src/fs_ficlone_linux.icpp +++ b/src/fs_ficlone_linux.icpp @@ -21,6 +21,7 @@ #include + namespace fs { int diff --git a/src/fs_ficlone_unsupported.icpp b/src/fs_ficlone_unsupported.icpp index aa6c4f0f..e7b864f5 100644 --- a/src/fs_ficlone_unsupported.icpp +++ b/src/fs_ficlone_unsupported.icpp @@ -18,6 +18,7 @@ #include "errno.hpp" + namespace fs { int diff --git a/src/fs_file_size.cpp b/src/fs_file_size.cpp index d06b7007..5528dbe1 100644 --- a/src/fs_file_size.cpp +++ b/src/fs_file_size.cpp @@ -18,7 +18,8 @@ #include "fs_fstat.hpp" -#include +#include + namespace fs { diff --git a/src/fs_file_size.hpp b/src/fs_file_size.hpp index 4977663e..1b01ea31 100644 --- a/src/fs_file_size.hpp +++ b/src/fs_file_size.hpp @@ -18,7 +18,8 @@ #pragma once -#include +#include + namespace fs { diff --git a/src/fs_findallfiles.cpp b/src/fs_findallfiles.cpp index a7a60d42..fc15f445 100644 --- a/src/fs_findallfiles.cpp +++ b/src/fs_findallfiles.cpp @@ -22,6 +22,7 @@ #include #include + namespace fs { void diff --git a/src/fs_findallfiles.hpp b/src/fs_findallfiles.hpp index ff3d81ed..db3e5895 100644 --- a/src/fs_findallfiles.hpp +++ b/src/fs_findallfiles.hpp @@ -18,13 +18,16 @@ #pragma once +#include "strvec.hpp" + #include #include + namespace fs { void - findallfiles(const std::vector &basepaths, - const char *fusepath, - std::vector *paths); + findallfiles(const StrVec &basepaths, + const char *fusepath, + StrVec *paths); } diff --git a/src/fs_findonfs.cpp b/src/fs_findonfs.cpp index e52c7333..ec8b2dc1 100644 --- a/src/fs_findonfs.cpp +++ b/src/fs_findonfs.cpp @@ -16,7 +16,7 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include "branch.hpp" +#include "branches.hpp" #include "errno.hpp" #include "fs_fstat.hpp" #include "fs_lstat.hpp" @@ -24,31 +24,29 @@ #include + namespace l { static int - findonfs(const BranchVec &branches_, - const std::string &fusepath_, - const int fd_, - std::string *basepath_) + findonfs(const Branches::CPtr &branches_, + const std::string &fusepath_, + const int fd_, + std::string *basepath_) { int rv; dev_t dev; struct stat st; std::string fullpath; - const Branch *branch; rv = fs::fstat(fd_,&st); if(rv == -1) return -1; dev = st.st_dev; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - fullpath = fs::path::make(branch->path,fusepath_); + fullpath = fs::path::make(branch.path,fusepath_); rv = fs::lstat(fullpath,&st); if(rv == -1) @@ -57,7 +55,7 @@ namespace l if(st.st_dev != dev) continue; - *basepath_ = branch->path; + *basepath_ = branch.path; return 0; } @@ -69,13 +67,11 @@ namespace l namespace fs { int - findonfs(const Branches &branches_, - const std::string &fusepath_, - const int fd_, - std::string *basepath_) + findonfs(const Branches::CPtr &branches_, + const std::string &fusepath_, + const int fd_, + std::string *basepath_) { - rwlock::ReadGuard guard(branches_.lock); - - return l::findonfs(branches_.vec,fusepath_,fd_,basepath_); + return l::findonfs(branches_,fusepath_,fd_,basepath_); } } diff --git a/src/fs_findonfs.hpp b/src/fs_findonfs.hpp index 3a72fd26..af7441a9 100644 --- a/src/fs_findonfs.hpp +++ b/src/fs_findonfs.hpp @@ -18,15 +18,16 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" #include + namespace fs { int - findonfs(const Branches &branches, - const std::string &fusepath, - const int fd, - std::string *basepath); + findonfs(const Branches::CPtr &branches, + const std::string &fusepath, + const int fd, + std::string *basepath); } diff --git a/src/fs_flistxattr.hpp b/src/fs_flistxattr.hpp index cbfa3425..1a9f6ae8 100644 --- a/src/fs_flistxattr.hpp +++ b/src/fs_flistxattr.hpp @@ -23,6 +23,7 @@ #include + namespace fs { static diff --git a/src/fs_flock.hpp b/src/fs_flock.hpp index 1cee9c6d..a6fd59f8 100644 --- a/src/fs_flock.hpp +++ b/src/fs_flock.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_fsetxattr.hpp b/src/fs_fsetxattr.hpp index d2cfb023..c2f2e129 100644 --- a/src/fs_fsetxattr.hpp +++ b/src/fs_fsetxattr.hpp @@ -25,6 +25,7 @@ #include + namespace fs { static diff --git a/src/fs_fstat.hpp b/src/fs_fstat.hpp index 233d72fb..702b1030 100644 --- a/src/fs_fstat.hpp +++ b/src/fs_fstat.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_fstatat.hpp b/src/fs_fstatat.hpp index 6148d9d5..cfada9fb 100644 --- a/src/fs_fstatat.hpp +++ b/src/fs_fstatat.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_fsync.hpp b/src/fs_fsync.hpp index c91e0181..91374487 100644 --- a/src/fs_fsync.hpp +++ b/src/fs_fsync.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_ftruncate.hpp b/src/fs_ftruncate.hpp index 0244bd2d..ec2c17a3 100644 --- a/src/fs_ftruncate.hpp +++ b/src/fs_ftruncate.hpp @@ -21,6 +21,7 @@ #include #include + namespace fs { static diff --git a/src/fs_futimens.hpp b/src/fs_futimens.hpp index 9f223303..2e4b8874 100644 --- a/src/fs_futimens.hpp +++ b/src/fs_futimens.hpp @@ -33,6 +33,7 @@ #include "fs_futimens_generic.hpp" #endif + namespace fs { static diff --git a/src/fs_futimens_freebsd_11.hpp b/src/fs_futimens_freebsd_11.hpp index 745bedee..d3e25247 100644 --- a/src/fs_futimens_freebsd_11.hpp +++ b/src/fs_futimens_freebsd_11.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_futimens_generic.hpp b/src/fs_futimens_generic.hpp index d6124151..ead03dc8 100644 --- a/src/fs_futimens_generic.hpp +++ b/src/fs_futimens_generic.hpp @@ -33,6 +33,7 @@ # define UTIME_OMIT ((1l << 30) - 2l) #endif + namespace l { static diff --git a/src/fs_futimens_linux.hpp b/src/fs_futimens_linux.hpp index 745bedee..d3e25247 100644 --- a/src/fs_futimens_linux.hpp +++ b/src/fs_futimens_linux.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_futimesat.hpp b/src/fs_futimesat.hpp index d18c44c4..f5a34d64 100644 --- a/src/fs_futimesat.hpp +++ b/src/fs_futimesat.hpp @@ -20,6 +20,7 @@ #include + namespace fs { int diff --git a/src/fs_futimesat_generic.icpp b/src/fs_futimesat_generic.icpp index 4f2b2134..acf49095 100644 --- a/src/fs_futimesat_generic.icpp +++ b/src/fs_futimesat_generic.icpp @@ -19,6 +19,7 @@ #include #include + namespace fs { int diff --git a/src/fs_futimesat_osx.icpp b/src/fs_futimesat_osx.icpp index e00c4ddd..b96b4623 100644 --- a/src/fs_futimesat_osx.icpp +++ b/src/fs_futimesat_osx.icpp @@ -26,6 +26,7 @@ #include #include + namespace l { static diff --git a/src/fs_getdents64.cpp b/src/fs_getdents64.cpp index a955fa12..0015cc07 100644 --- a/src/fs_getdents64.cpp +++ b/src/fs_getdents64.cpp @@ -23,6 +23,7 @@ #include #endif + namespace fs { int diff --git a/src/fs_getdents64.hpp b/src/fs_getdents64.hpp index e1c12d6a..f328c431 100644 --- a/src/fs_getdents64.hpp +++ b/src/fs_getdents64.hpp @@ -18,6 +18,7 @@ #pragma once + namespace fs { int diff --git a/src/fs_getfl.cpp b/src/fs_getfl.cpp index 891561db..f84676a6 100644 --- a/src/fs_getfl.cpp +++ b/src/fs_getfl.cpp @@ -18,6 +18,7 @@ #include + namespace fs { int diff --git a/src/fs_getfl.hpp b/src/fs_getfl.hpp index ddb30034..0abb3e20 100644 --- a/src/fs_getfl.hpp +++ b/src/fs_getfl.hpp @@ -18,6 +18,7 @@ #pragma once + namespace fs { int getfl(const int fd); diff --git a/src/fs_glob.cpp b/src/fs_glob.cpp index 02efe6c5..fb7177dd 100644 --- a/src/fs_glob.cpp +++ b/src/fs_glob.cpp @@ -14,16 +14,16 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include - #include +#include #include #include using std::string; using std::vector; + namespace fs { void diff --git a/src/fs_glob.hpp b/src/fs_glob.hpp index ed16c95c..9d2b3b06 100644 --- a/src/fs_glob.hpp +++ b/src/fs_glob.hpp @@ -19,6 +19,7 @@ #include #include + namespace fs { void diff --git a/src/fs_has_space.cpp b/src/fs_has_space.cpp index f41ee8b5..3ce5bc18 100644 --- a/src/fs_has_space.cpp +++ b/src/fs_has_space.cpp @@ -19,9 +19,9 @@ #include "fs_statvfs.hpp" #include "statvfs_util.hpp" +#include #include -#include namespace fs { diff --git a/src/fs_has_space.hpp b/src/fs_has_space.hpp index ed4c5c79..64aa7825 100644 --- a/src/fs_has_space.hpp +++ b/src/fs_has_space.hpp @@ -18,9 +18,9 @@ #pragma once +#include #include -#include namespace fs { diff --git a/src/fs_info.cpp b/src/fs_info.cpp index e36d41f7..6368036e 100644 --- a/src/fs_info.cpp +++ b/src/fs_info.cpp @@ -23,12 +23,12 @@ #include "fs_statvfs_cache.hpp" #include "statvfs_util.hpp" -#include - +#include #include using std::string; + namespace fs { int diff --git a/src/fs_info.hpp b/src/fs_info.hpp index cc8639a3..ec7cf30d 100644 --- a/src/fs_info.hpp +++ b/src/fs_info.hpp @@ -22,6 +22,7 @@ #include + namespace fs { int diff --git a/src/fs_info_t.hpp b/src/fs_info_t.hpp index 623617d7..7db05d74 100644 --- a/src/fs_info_t.hpp +++ b/src/fs_info_t.hpp @@ -18,7 +18,8 @@ #pragma once -#include +#include + namespace fs { diff --git a/src/fs_inode.cpp b/src/fs_inode.cpp index 20256940..b0330c77 100644 --- a/src/fs_inode.cpp +++ b/src/fs_inode.cpp @@ -21,10 +21,10 @@ #include "fs_inode.hpp" #include "wyhash.h" +#include #include #include -#include #include #include @@ -184,10 +184,16 @@ namespace fs return "passthrough"; if(g_func == path_hash) return "path-hash"; + if(g_func == path_hash32) + return "path-hash32"; if(g_func == devino_hash) return "devino-hash"; + if(g_func == devino_hash32) + return "devino-hash32"; if(g_func == hybrid_hash) return "hybrid-hash"; + if(g_func == hybrid_hash32) + return "hybrid-hash32"; return std::string(); } diff --git a/src/fs_inode.hpp b/src/fs_inode.hpp index a7e0fb76..baf7d718 100644 --- a/src/fs_inode.hpp +++ b/src/fs_inode.hpp @@ -18,9 +18,9 @@ #pragma once +#include #include -#include #include diff --git a/src/fs_ioctl.hpp b/src/fs_ioctl.hpp index cdb74e2b..86ee9bdf 100644 --- a/src/fs_ioctl.hpp +++ b/src/fs_ioctl.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_lchmod.hpp b/src/fs_lchmod.hpp index 21c94500..95012ab0 100644 --- a/src/fs_lchmod.hpp +++ b/src/fs_lchmod.hpp @@ -26,6 +26,7 @@ #define MODE_BITS (S_ISUID|S_ISGID|S_ISVTX|S_IRWXU|S_IRWXG|S_IRWXO) + namespace fs { static diff --git a/src/fs_lchown.hpp b/src/fs_lchown.hpp index 83fa30d8..6091a269 100644 --- a/src/fs_lchown.hpp +++ b/src/fs_lchown.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_lgetxattr.hpp b/src/fs_lgetxattr.hpp index 482fa35c..7d4fe8e1 100644 --- a/src/fs_lgetxattr.hpp +++ b/src/fs_lgetxattr.hpp @@ -25,6 +25,7 @@ #include + namespace fs { static diff --git a/src/fs_link.hpp b/src/fs_link.hpp index 09ef6459..8b41d4a2 100644 --- a/src/fs_link.hpp +++ b/src/fs_link.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_llistxattr.hpp b/src/fs_llistxattr.hpp index e8c83dc4..9818a665 100644 --- a/src/fs_llistxattr.hpp +++ b/src/fs_llistxattr.hpp @@ -25,6 +25,7 @@ #include + namespace fs { static diff --git a/src/fs_lremovexattr.hpp b/src/fs_lremovexattr.hpp index 4d9f3f53..06258f9d 100644 --- a/src/fs_lremovexattr.hpp +++ b/src/fs_lremovexattr.hpp @@ -23,6 +23,7 @@ #include + namespace fs { static diff --git a/src/fs_lseek.hpp b/src/fs_lseek.hpp index 34744c49..79bcfe3b 100644 --- a/src/fs_lseek.hpp +++ b/src/fs_lseek.hpp @@ -21,6 +21,7 @@ #include #include + namespace fs { static diff --git a/src/fs_lsetxattr.hpp b/src/fs_lsetxattr.hpp index 41baf862..ba3a7325 100644 --- a/src/fs_lsetxattr.hpp +++ b/src/fs_lsetxattr.hpp @@ -25,6 +25,7 @@ #include + namespace fs { static diff --git a/src/fs_lstat.hpp b/src/fs_lstat.hpp index 83254b83..dd8e8384 100644 --- a/src/fs_lstat.hpp +++ b/src/fs_lstat.hpp @@ -24,6 +24,7 @@ #include #include + namespace fs { static diff --git a/src/fs_lutimens.hpp b/src/fs_lutimens.hpp index ab1ce218..e2937a6d 100644 --- a/src/fs_lutimens.hpp +++ b/src/fs_lutimens.hpp @@ -21,6 +21,7 @@ #include "fs_utimensat.hpp" #include "fs_stat_utils.hpp" + namespace fs { static diff --git a/src/fs_mkdir.hpp b/src/fs_mkdir.hpp index 05eb78b5..8d55c595 100644 --- a/src/fs_mkdir.hpp +++ b/src/fs_mkdir.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_mknod.hpp b/src/fs_mknod.hpp index bf85f3b2..9bf80636 100644 --- a/src/fs_mknod.hpp +++ b/src/fs_mknod.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_mktemp.cpp b/src/fs_mktemp.cpp index 1f2166eb..bd4ea2cf 100644 --- a/src/fs_mktemp.cpp +++ b/src/fs_mktemp.cpp @@ -29,6 +29,7 @@ using std::string; #define PADLEN 6 #define MAX_ATTEMPTS 10 + static string generate_tmp_path(const string &base_) diff --git a/src/fs_mktemp.hpp b/src/fs_mktemp.hpp index 2bd19b16..aecf0e74 100644 --- a/src/fs_mktemp.hpp +++ b/src/fs_mktemp.hpp @@ -20,6 +20,7 @@ #include + namespace fs { int diff --git a/src/fs_movefile.cpp b/src/fs_movefile.cpp index 2c2c13e2..bd9c37b6 100644 --- a/src/fs_movefile.cpp +++ b/src/fs_movefile.cpp @@ -43,12 +43,13 @@ using std::string; using std::vector; + namespace l { static int - movefile(Policy::Func::Create createFunc_, - const Branches &branches_, + movefile(const Policy::Create &createFunc_, + const Branches::CPtr &branches_, const string &fusepath_, int *origfd_) { @@ -132,19 +133,19 @@ namespace l namespace fs { int - movefile(const Policy *policy_, - const Branches &basepaths_, - const string &fusepath_, - int *origfd_) + movefile(const Policy::Create &policy_, + const Branches::CPtr &basepaths_, + const string &fusepath_, + int *origfd_) { return l::movefile(policy_,basepaths_,fusepath_,origfd_); } int - movefile_as_root(const Policy *policy_, - const Branches &basepaths_, - const string &fusepath_, - int *origfd_) + movefile_as_root(const Policy::Create &policy_, + const Branches::CPtr &basepaths_, + const string &fusepath_, + int *origfd_) { const ugid::Set ugid(0,0); diff --git a/src/fs_movefile.hpp b/src/fs_movefile.hpp index 7b21312e..8df1e1c2 100644 --- a/src/fs_movefile.hpp +++ b/src/fs_movefile.hpp @@ -16,22 +16,23 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" #include "policy.hpp" #include + namespace fs { int - movefile(const Policy *policy, - const Branches &branches, - const std::string &fusepath, - int *origfd); + movefile(const Policy::Create &policy, + const Branches::CPtr &branches, + const std::string &fusepath, + int *origfd); int - movefile_as_root(const Policy *policy, - const Branches &branches, - const std::string &fusepath, - int *origfd); + movefile_as_root(const Policy::Create &policy, + const Branches::CPtr &branches, + const std::string &fusepath, + int *origfd); } diff --git a/src/fs_open.hpp b/src/fs_open.hpp index 2e4a7279..02364361 100644 --- a/src/fs_open.hpp +++ b/src/fs_open.hpp @@ -24,6 +24,7 @@ #include #include + namespace fs { static diff --git a/src/fs_opendir.hpp b/src/fs_opendir.hpp index 7b62aecf..a7f1197e 100644 --- a/src/fs_opendir.hpp +++ b/src/fs_opendir.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_path.cpp b/src/fs_path.cpp index 861973f3..90c65a5c 100644 --- a/src/fs_path.cpp +++ b/src/fs_path.cpp @@ -14,17 +14,18 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include "fs_path.hpp" + +#include + #include #include #include #include -#include - -#include "fs_path.hpp" - using std::string; + namespace fs { namespace path diff --git a/src/fs_path.hpp b/src/fs_path.hpp index 531c21df..d36b8fd6 100644 --- a/src/fs_path.hpp +++ b/src/fs_path.hpp @@ -19,6 +19,7 @@ #include #include + namespace fs { namespace path diff --git a/src/fs_read.hpp b/src/fs_read.hpp index d794543e..c1021fe7 100644 --- a/src/fs_read.hpp +++ b/src/fs_read.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_readdir.hpp b/src/fs_readdir.hpp index 7a82d61e..cad0d643 100644 --- a/src/fs_readdir.hpp +++ b/src/fs_readdir.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_readlink.hpp b/src/fs_readlink.hpp index a8fc1a5b..0cd2df0f 100644 --- a/src/fs_readlink.hpp +++ b/src/fs_readlink.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_realpath.hpp b/src/fs_realpath.hpp index d31a270c..31c30316 100644 --- a/src/fs_realpath.hpp +++ b/src/fs_realpath.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_realpathize.cpp b/src/fs_realpathize.cpp index 42db33b6..fe85d2d2 100644 --- a/src/fs_realpathize.cpp +++ b/src/fs_realpathize.cpp @@ -21,6 +21,7 @@ #include #include + namespace fs { void diff --git a/src/fs_realpathize.hpp b/src/fs_realpathize.hpp index dfd6270d..0702097e 100644 --- a/src/fs_realpathize.hpp +++ b/src/fs_realpathize.hpp @@ -21,6 +21,7 @@ #include #include + namespace fs { void diff --git a/src/fs_remove.hpp b/src/fs_remove.hpp index 07ae25cc..b9d7f417 100644 --- a/src/fs_remove.hpp +++ b/src/fs_remove.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_rename.hpp b/src/fs_rename.hpp index 7bfdba7e..a0fe92b3 100644 --- a/src/fs_rename.hpp +++ b/src/fs_rename.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_rmdir.hpp b/src/fs_rmdir.hpp index d87a3f42..dd39a5b4 100644 --- a/src/fs_rmdir.hpp +++ b/src/fs_rmdir.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_sendfile.hpp b/src/fs_sendfile.hpp index 54238426..ae2fb017 100644 --- a/src/fs_sendfile.hpp +++ b/src/fs_sendfile.hpp @@ -16,6 +16,7 @@ #pragma once + namespace fs { ssize_t diff --git a/src/fs_sendfile_linux.icpp b/src/fs_sendfile_linux.icpp index a6291ddc..5f62c70a 100644 --- a/src/fs_sendfile_linux.icpp +++ b/src/fs_sendfile_linux.icpp @@ -14,9 +14,10 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include "errno.hpp" + #include -#include "errno.hpp" namespace fs { diff --git a/src/fs_sendfile_unsupported.icpp b/src/fs_sendfile_unsupported.icpp index a08fc1a4..96a8827a 100644 --- a/src/fs_sendfile_unsupported.icpp +++ b/src/fs_sendfile_unsupported.icpp @@ -14,9 +14,10 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include "errno.hpp" + #include -#include "errno.hpp" namespace fs { diff --git a/src/fs_setfl.cpp b/src/fs_setfl.cpp index faf1b2e4..7251c66b 100644 --- a/src/fs_setfl.cpp +++ b/src/fs_setfl.cpp @@ -18,6 +18,7 @@ #include + namespace fs { int diff --git a/src/fs_setfl.hpp b/src/fs_setfl.hpp index 9b146f10..31c07f71 100644 --- a/src/fs_setfl.hpp +++ b/src/fs_setfl.hpp @@ -20,6 +20,7 @@ #include + namespace fs { int diff --git a/src/fs_stat.hpp b/src/fs_stat.hpp index 5b5fed7c..b1c7c864 100644 --- a/src/fs_stat.hpp +++ b/src/fs_stat.hpp @@ -24,6 +24,7 @@ #include #include + namespace fs { static diff --git a/src/fs_stat_utils.hpp b/src/fs_stat_utils.hpp index 42f13dd8..777a7140 100644 --- a/src/fs_stat_utils.hpp +++ b/src/fs_stat_utils.hpp @@ -24,6 +24,7 @@ #include #include + namespace fs { static diff --git a/src/fs_statvfs.hpp b/src/fs_statvfs.hpp index 26f8a054..db0c1cf1 100644 --- a/src/fs_statvfs.hpp +++ b/src/fs_statvfs.hpp @@ -22,6 +22,7 @@ #include "fs_close.hpp" #include "fs_open.hpp" +#include #include #include @@ -30,6 +31,7 @@ # define O_PATH 0 #endif + namespace fs { static diff --git a/src/fs_statvfs_cache.cpp b/src/fs_statvfs_cache.cpp index 4cea84d0..a0c73192 100644 --- a/src/fs_statvfs_cache.cpp +++ b/src/fs_statvfs_cache.cpp @@ -19,14 +19,15 @@ #include "fs_statvfs.hpp" #include "statvfs_util.hpp" +#include #include #include #include -#include #include #include + struct Element { uint64_t time; diff --git a/src/fs_statvfs_cache.hpp b/src/fs_statvfs_cache.hpp index 1beb46f3..d48d075d 100644 --- a/src/fs_statvfs_cache.hpp +++ b/src/fs_statvfs_cache.hpp @@ -18,9 +18,11 @@ #pragma once -#include +#include + #include + namespace fs { uint64_t diff --git a/src/fs_symlink.hpp b/src/fs_symlink.hpp index 6fa9e535..1db1823e 100644 --- a/src/fs_symlink.hpp +++ b/src/fs_symlink.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_truncate.hpp b/src/fs_truncate.hpp index 512214cc..9a88ba60 100644 --- a/src/fs_truncate.hpp +++ b/src/fs_truncate.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_unlink.hpp b/src/fs_unlink.hpp index 08828595..5fad40d2 100644 --- a/src/fs_unlink.hpp +++ b/src/fs_unlink.hpp @@ -22,6 +22,7 @@ #include + namespace fs { static diff --git a/src/fs_utimensat_freebsd.hpp b/src/fs_utimensat_freebsd.hpp index ef3935ff..251c3f56 100644 --- a/src/fs_utimensat_freebsd.hpp +++ b/src/fs_utimensat_freebsd.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_utimensat_generic.hpp b/src/fs_utimensat_generic.hpp index 1834c0c3..2bd5bf26 100644 --- a/src/fs_utimensat_generic.hpp +++ b/src/fs_utimensat_generic.hpp @@ -36,6 +36,7 @@ # define UTIME_OMIT ((1l << 30) - 2l) #endif + namespace l { static diff --git a/src/fs_utimensat_linux.hpp b/src/fs_utimensat_linux.hpp index ef3935ff..251c3f56 100644 --- a/src/fs_utimensat_linux.hpp +++ b/src/fs_utimensat_linux.hpp @@ -23,6 +23,7 @@ #include #include + namespace fs { static diff --git a/src/fs_write.hpp b/src/fs_write.hpp index 96eebbb1..750bef03 100644 --- a/src/fs_write.hpp +++ b/src/fs_write.hpp @@ -20,6 +20,7 @@ #include + namespace fs { static diff --git a/src/fs_xattr.cpp b/src/fs_xattr.cpp index c73766d6..e7ac84a3 100644 --- a/src/fs_xattr.cpp +++ b/src/fs_xattr.cpp @@ -31,12 +31,12 @@ #include #include - using std::string; using std::vector; using std::map; using std::istringstream; + namespace fs { namespace xattr diff --git a/src/fs_xattr.hpp b/src/fs_xattr.hpp index caad0749..da7d2ba3 100644 --- a/src/fs_xattr.hpp +++ b/src/fs_xattr.hpp @@ -20,6 +20,7 @@ #include #include + namespace fs { namespace xattr diff --git a/src/func.cpp b/src/func.cpp index b82773ab..39aa7194 100644 --- a/src/func.cpp +++ b/src/func.cpp @@ -18,22 +18,45 @@ #include "func.hpp" + int -Func::from_string(const std::string &s_) +Func::Base::Action::from_string(const std::string &policyname_) +{ + policy = Policies::Action::find(policyname_); + + return 0; +} + +std::string +Func::Base::Action::to_string(void) const { - const Policy *tmp; + return policy.name(); +} - tmp = &Policy::find(s_); - if(tmp == Policy::invalid) - return -EINVAL; +int +Func::Base::Create::from_string(const std::string &policyname_) +{ + policy = Policies::Create::find(policyname_); - policy = tmp; + return 0; +} + +std::string +Func::Base::Create::to_string(void) const +{ + return policy.name(); +} + +int +Func::Base::Search::from_string(const std::string &policyname_) +{ + policy = Policies::Search::find(policyname_); return 0; } std::string -Func::to_string(void) const +Func::Base::Search::to_string(void) const { - return policy->to_string(); + return policy.name(); } diff --git a/src/func.hpp b/src/func.hpp index f1a390f3..97ef6f64 100644 --- a/src/func.hpp +++ b/src/func.hpp @@ -19,203 +19,166 @@ #pragma once #include "policy.hpp" +#include "policies.hpp" #include "tofrom_string.hpp" #include -#include -class Func : public ToFromString + +namespace Func { -public: - Func(const Policy &policy_) - : policy(&policy_) + namespace Base { - } + class Action : public ToFromString + { + public: + Action(Policy::ActionImpl *policy_) + : policy(policy_) + {} -public: - int from_string(const std::string &s); - std::string to_string() const; + public: + int from_string(const std::string &s) final; + std::string to_string() const final; -public: - const Policy *policy; -}; + public: + Policy::Action policy; + }; -class FuncAccess : public Func -{ -public: - FuncAccess() - : Func(Policy::ff) - { + class Create : public ToFromString + { + public: + Create(Policy::CreateImpl *policy_) + : policy(policy_) + {} + + public: + int from_string(const std::string &s) final; + std::string to_string() const final; + + public: + Policy::Create policy; + }; + + class Search : public ToFromString + { + public: + Search(Policy::SearchImpl *policy_) + : policy(policy_) + {} + + public: + int from_string(const std::string &s) final; + std::string to_string() const final; + + public: + Policy::Search policy; + }; + + class ActionDefault : public Action + { + public: + ActionDefault() + : Func::Base::Action(&Policies::Action::epall) + { + } + }; + + class CreateDefault : public Create + { + public: + CreateDefault() + : Func::Base::Create(&Policies::Create::epmfs) + { + } + }; + + class SearchDefault : public Search + { + public: + SearchDefault() + : Func::Base::Search(&Policies::Search::ff) + { + } + }; } -}; -class FuncChmod : public Func -{ -public: - FuncChmod() - : Func(Policy::epall) + class Access final : public Base::SearchDefault { - } -}; + }; -class FuncChown : public Func -{ -public: - FuncChown() - : Func(Policy::epall) + class Chmod final : public Base::ActionDefault { - } -}; + }; -class FuncCreate : public Func -{ -public: - FuncCreate() - : Func(Policy::epmfs) + class Chown final : public Base::ActionDefault { - } -}; + }; -class FuncGetAttr : public Func -{ -public: - FuncGetAttr() - : Func(Policy::ff) + class Create final : public Base::CreateDefault { - } -}; + }; -class FuncGetXAttr : public Func -{ -public: - FuncGetXAttr() - : Func(Policy::ff) + class GetAttr final : public Base::SearchDefault { - } -}; + }; -class FuncLink : public Func -{ -public: - FuncLink() - : Func(Policy::epall) + class GetXAttr final : public Base::SearchDefault { - } -}; + }; -class FuncListXAttr : public Func -{ -public: - FuncListXAttr() - : Func(Policy::ff) + class Link final : public Base::ActionDefault { - } -}; + }; -class FuncMkdir : public Func -{ -public: - FuncMkdir() - : Func(Policy::epmfs) + class ListXAttr final : public Base::SearchDefault { - } -}; + }; -class FuncMknod : public Func -{ -public: - FuncMknod() - : Func(Policy::epmfs) + class Mkdir final : public Base::CreateDefault { - } -}; + }; -class FuncOpen : public Func -{ -public: - FuncOpen() - : Func(Policy::ff) + class Mknod final : public Base::CreateDefault { - } -}; + }; -class FuncReadlink : public Func -{ -public: - FuncReadlink() - : Func(Policy::ff) + class Open final : public Base::SearchDefault { - } -}; + }; -class FuncRemoveXAttr : public Func -{ -public: - FuncRemoveXAttr() - : Func(Policy::epall) + class Readlink final : public Base::SearchDefault { - } -}; + }; -class FuncRename : public Func -{ -public: - FuncRename() - : Func(Policy::epall) + class RemoveXAttr final : public Base::ActionDefault { - } -}; + }; -class FuncRmdir : public Func -{ -public: - FuncRmdir() - : Func(Policy::epall) + class Rename final : public Base::ActionDefault { - } -}; + }; -class FuncSetXAttr : public Func -{ -public: - FuncSetXAttr() - : Func(Policy::epall) + class Rmdir final : public Base::ActionDefault { - } -}; + }; -class FuncSymlink : public Func -{ -public: - FuncSymlink() - : Func(Policy::epmfs) + class SetXAttr final : public Base::ActionDefault { - } -}; + }; -class FuncTruncate : public Func -{ -public: - FuncTruncate() - : Func(Policy::epall) + class Symlink final : public Base::CreateDefault { - } -}; + }; -class FuncUnlink : public Func -{ -public: - FuncUnlink() - : Func(Policy::epall) + class Truncate final : public Base::ActionDefault { - } -}; + }; -class FuncUtimens : public Func -{ -public: - FuncUtimens() - : Func(Policy::epall) + class Unlink final : public Base::ActionDefault { - } -}; + }; + + class Utimens final : public Base::ActionDefault + { + }; +} diff --git a/src/func_category.cpp b/src/func_category.cpp deleted file mode 100644 index eb60c9d3..00000000 --- a/src/func_category.cpp +++ /dev/null @@ -1,90 +0,0 @@ -/* - ISC License - - Copyright (c) 2019, Antonio SJ Musumeci - - Permission to use, copy, modify, and/or distribute this software for any - purpose with or without fee is hereby granted, provided that the above - copyright notice and this permission notice appear in all copies. - - THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES - WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF - MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR - ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES - WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN - ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF - OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. -*/ - -#include "func_category.hpp" -#include "str.hpp" - -#include "buildvector.hpp" - -#include -#include -#include - -int -FuncCategory::from_string(const std::string &s_) -{ - const Policy *tmp; - - tmp = &Policy::find(s_); - if(tmp == &Policy::invalid) - return -EINVAL; - - for(uint64_t i = 0; i < _funcs.size(); i++) - _funcs[i]->policy = tmp; - - return 0; -} - -std::string -FuncCategory::to_string(void) const -{ - std::vector rv; - - for(uint64_t i = 0; i < _funcs.size(); i++) - rv.push_back(_funcs[i]->policy->to_string()); - - std::sort(rv.begin(),rv.end()); - rv.erase(std::unique(rv.begin(),rv.end()),rv.end()); - - return str::join(rv,','); -} - -FuncCategoryCreate::FuncCategoryCreate(Funcs &funcs_) -{ - _funcs = buildvector - (&funcs_.create) - (&funcs_.mkdir) - (&funcs_.mknod) - (&funcs_.symlink); -} - -FuncCategoryAction::FuncCategoryAction(Funcs &funcs_) -{ - _funcs = buildvector - (&funcs_.chmod) - (&funcs_.chown) - (&funcs_.link) - (&funcs_.removexattr) - (&funcs_.rename) - (&funcs_.rmdir) - (&funcs_.setxattr) - (&funcs_.truncate) - (&funcs_.unlink) - (&funcs_.utimens); -} - -FuncCategorySearch::FuncCategorySearch(Funcs &funcs_) -{ - _funcs = buildvector - (&funcs_.access) - (&funcs_.getattr) - (&funcs_.getxattr) - (&funcs_.listxattr) - (&funcs_.open) - (&funcs_.readlink); -} diff --git a/src/func_category.hpp b/src/func_category.hpp deleted file mode 100644 index 775239f5..00000000 --- a/src/func_category.hpp +++ /dev/null @@ -1,69 +0,0 @@ -/* - ISC License - - Copyright (c) 2019, Antonio SJ Musumeci - - Permission to use, copy, modify, and/or distribute this software for any - purpose with or without fee is hereby granted, provided that the above - copyright notice and this permission notice appear in all copies. - - THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES - WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF - MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR - ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES - WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN - ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF - OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. -*/ - -#pragma once - -#include "funcs.hpp" -#include "tofrom_string.hpp" - -#include -#include - -class FuncCategory : public ToFromString -{ -public: - int from_string(const std::string &); - std::string to_string(void) const; - -protected: - std::vector _funcs; -}; - -class FuncCategoryCreate : public FuncCategory -{ -public: - FuncCategoryCreate(Funcs &funcs_); -}; - -class FuncCategoryAction : public FuncCategory -{ -public: - FuncCategoryAction(Funcs &funcs_); -}; - -class FuncCategorySearch : public FuncCategory -{ -public: - FuncCategorySearch(Funcs &funcs_); -}; - -class FuncCategories -{ -public: - FuncCategories(Funcs &funcs_) - : action(funcs_), - create(funcs_), - search(funcs_) - { - } - -public: - FuncCategoryAction action; - FuncCategoryCreate create; - FuncCategorySearch search; -}; diff --git a/src/funcs.hpp b/src/funcs.hpp index a482b36e..dbb1eb8c 100644 --- a/src/funcs.hpp +++ b/src/funcs.hpp @@ -20,27 +20,27 @@ #include "func.hpp" -class Funcs + +struct Funcs { -public: - FuncAccess access; - FuncChmod chmod; - FuncChown chown; - FuncCreate create; - FuncGetAttr getattr; - FuncGetXAttr getxattr; - FuncLink link; - FuncListXAttr listxattr; - FuncMkdir mkdir; - FuncMknod mknod; - FuncOpen open; - FuncReadlink readlink; - FuncRemoveXAttr removexattr; - FuncRename rename; - FuncRmdir rmdir; - FuncSetXAttr setxattr; - FuncSymlink symlink; - FuncTruncate truncate; - FuncUnlink unlink; - FuncUtimens utimens; + Func::Access access; + Func::Chmod chmod; + Func::Chown chown; + Func::Create create; + Func::GetAttr getattr; + Func::GetXAttr getxattr; + Func::Link link; + Func::ListXAttr listxattr; + Func::Mkdir mkdir; + Func::Mknod mknod; + Func::Open open; + Func::Readlink readlink; + Func::RemoveXAttr removexattr; + Func::Rename rename; + Func::Rmdir rmdir; + Func::SetXAttr setxattr; + Func::Symlink symlink; + Func::Truncate truncate; + Func::Unlink unlink; + Func::Utimens utimens; }; diff --git a/src/fuse_access.cpp b/src/fuse_access.cpp index e9705ce3..6e196295 100644 --- a/src/fuse_access.cpp +++ b/src/fuse_access.cpp @@ -26,26 +26,27 @@ using std::string; using std::vector; + namespace l { static int - access(Policy::Func::Search searchFunc, + access(const Policy::Search &searchFunc_, const Branches &branches_, - const char *fusepath, - const int mask) + const char *fusepath_, + const int mask_) { int rv; string fullpath; - vector basepaths; + StrVec basepaths; - rv = searchFunc(branches_,fusepath,&basepaths); + rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; - fullpath = fs::path::make(basepaths[0],fusepath); + fullpath = fs::path::make(basepaths[0],fusepath_); - rv = fs::eaccess(fullpath,mask); + rv = fs::eaccess(fullpath,mask_); return ((rv == -1) ? -errno : 0); } @@ -54,16 +55,16 @@ namespace l namespace FUSE { int - access(const char *fusepath, - int mask) + access(const char *fusepath_, + int mask_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::access(config.func.access.policy, - config.branches, - fusepath, - mask); + return l::access(cfg->func.access.policy, + cfg->branches, + fusepath_, + mask_); } } diff --git a/src/fuse_access.hpp b/src/fuse_access.hpp index 7fc64277..72386aae 100644 --- a/src/fuse_access.hpp +++ b/src/fuse_access.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_chmod.cpp b/src/fuse_chmod.cpp index 938427b9..c2e0e709 100644 --- a/src/fuse_chmod.cpp +++ b/src/fuse_chmod.cpp @@ -24,12 +24,10 @@ #include "fuse.h" #include -#include #include using std::string; -using std::vector; namespace l @@ -73,10 +71,10 @@ namespace l static void - chmod_loop(const vector &basepaths_, - const char *fusepath_, - const mode_t mode_, - PolicyRV *prv_) + chmod_loop(const StrVec &basepaths_, + const char *fusepath_, + const mode_t mode_, + PolicyRV *prv_) { for(size_t i = 0, ei = basepaths_.size(); i != ei; i++) { @@ -86,15 +84,15 @@ namespace l static int - chmod(Policy::Func::Action actionFunc_, - Policy::Func::Search searchFunc_, + chmod(const Policy::Action &actionFunc_, + const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const mode_t mode_) { int rv; PolicyRV prv; - vector basepaths; + StrVec basepaths; rv = actionFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -121,13 +119,13 @@ namespace FUSE chmod(const char *fusepath_, mode_t mode_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::chmod(config.func.chmod.policy, - config.func.getattr.policy, - config.branches, + return l::chmod(cfg->func.chmod.policy, + cfg->func.getattr.policy, + cfg->branches, fusepath_, mode_); } diff --git a/src/fuse_chmod.hpp b/src/fuse_chmod.hpp index 6bbcdb40..ea335a34 100644 --- a/src/fuse_chmod.hpp +++ b/src/fuse_chmod.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_chown.cpp b/src/fuse_chown.cpp index 70788687..106545b4 100644 --- a/src/fuse_chown.cpp +++ b/src/fuse_chown.cpp @@ -21,7 +21,7 @@ #include "policy_rv.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -86,8 +86,8 @@ namespace l static int - chown(Policy::Func::Action actionFunc_, - Policy::Func::Search searchFunc_, + chown(const Policy::Action &actionFunc_, + const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const uid_t uid_, @@ -123,13 +123,13 @@ namespace FUSE uid_t uid_, gid_t gid_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::chown(config.func.chown.policy, - config.func.getattr.policy, - config.branches, + return l::chown(cfg->func.chown.policy, + cfg->func.getattr.policy, + cfg->branches, fusepath_, uid_, gid_); diff --git a/src/fuse_chown.hpp b/src/fuse_chown.hpp index cda0058d..1a367acf 100644 --- a/src/fuse_chown.hpp +++ b/src/fuse_chown.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_copy_file_range.cpp b/src/fuse_copy_file_range.cpp index 2c88733e..34988d86 100644 --- a/src/fuse_copy_file_range.cpp +++ b/src/fuse_copy_file_range.cpp @@ -18,10 +18,11 @@ #include "fileinfo.hpp" #include "fs_copy_file_range.hpp" -#include +#include "fuse.h" #include + namespace l { static diff --git a/src/fuse_copy_file_range.hpp b/src/fuse_copy_file_range.hpp index 30185ad3..ef18a3e6 100644 --- a/src/fuse_copy_file_range.hpp +++ b/src/fuse_copy_file_range.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { ssize_t diff --git a/src/fuse_create.cpp b/src/fuse_create.cpp index 4a854d1a..5d66d5eb 100644 --- a/src/fuse_create.cpp +++ b/src/fuse_create.cpp @@ -23,7 +23,7 @@ #include "fs_path.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -31,6 +31,7 @@ using std::string; using std::vector; + namespace l { /* @@ -54,15 +55,15 @@ namespace l static void - config_to_ffi_flags(const Config &config_, + config_to_ffi_flags(Config::Read &cfg_, fuse_file_info_t *ffi_) { - switch(config_.cache_files) + switch(cfg_->cache_files) { case CacheFiles::ENUM::LIBFUSE: - ffi_->direct_io = config_.direct_io; - ffi_->keep_cache = config_.kernel_cache; - ffi_->auto_cache = config_.auto_cache; + ffi_->direct_io = cfg_->direct_io; + ffi_->keep_cache = cfg_->kernel_cache; + ffi_->auto_cache = cfg_->auto_cache; break; case CacheFiles::ENUM::OFF: ffi_->direct_io = 1; @@ -125,8 +126,8 @@ namespace l static int - create(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + create(const Policy::Search &searchFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const char *fusepath_, const mode_t mode_, @@ -137,8 +138,8 @@ namespace l int rv; string fullpath; string fusedirpath; - vector createpaths; - vector existingpaths; + StrVec createpaths; + StrVec existingpaths; fusedirpath = fs::path::dirname(fusepath_); @@ -170,18 +171,18 @@ namespace FUSE mode_t mode_, fuse_file_info_t *ffi_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - l::config_to_ffi_flags(config,ffi_); + l::config_to_ffi_flags(cfg,ffi_); - if(config.writeback_cache) + if(cfg->writeback_cache) l::tweak_flags_writeback_cache(&ffi_->flags); - return l::create(config.func.getattr.policy, - config.func.create.policy, - config.branches, + return l::create(cfg->func.getattr.policy, + cfg->func.create.policy, + cfg->branches, fusepath_, mode_, fc->umask, diff --git a/src/fuse_create.hpp b/src/fuse_create.hpp index 0fa08a80..36a74d17 100644 --- a/src/fuse_create.hpp +++ b/src/fuse_create.hpp @@ -16,10 +16,11 @@ #pragma once -#include +#include "fuse.h" #include + namespace FUSE { int diff --git a/src/fuse_destroy.hpp b/src/fuse_destroy.hpp index 60ae8577..31410ee3 100644 --- a/src/fuse_destroy.hpp +++ b/src/fuse_destroy.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { void diff --git a/src/fuse_fallocate.cpp b/src/fuse_fallocate.cpp index 8235ce43..c1b4e82d 100644 --- a/src/fuse_fallocate.cpp +++ b/src/fuse_fallocate.cpp @@ -18,7 +18,8 @@ #include "fileinfo.hpp" #include "fs_fallocate.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_fallocate.hpp b/src/fuse_fallocate.hpp index f29e4a7f..978cc032 100644 --- a/src/fuse_fallocate.hpp +++ b/src/fuse_fallocate.hpp @@ -18,6 +18,7 @@ #include "fuse.h" + namespace FUSE { int diff --git a/src/fuse_fchmod.cpp b/src/fuse_fchmod.cpp index 3b8cdf8c..27d802de 100644 --- a/src/fuse_fchmod.cpp +++ b/src/fuse_fchmod.cpp @@ -18,7 +18,8 @@ #include "fileinfo.hpp" #include "fs_fchmod.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_fchmod.hpp b/src/fuse_fchmod.hpp index 96d4be7e..64f9e7a5 100644 --- a/src/fuse_fchmod.hpp +++ b/src/fuse_fchmod.hpp @@ -16,10 +16,11 @@ #pragma once -#include +#include "fuse.h" #include + namespace FUSE { int diff --git a/src/fuse_fchown.cpp b/src/fuse_fchown.cpp index 5bf873ab..48e8aa16 100644 --- a/src/fuse_fchown.cpp +++ b/src/fuse_fchown.cpp @@ -18,10 +18,11 @@ #include "fileinfo.hpp" #include "fs_fchown.hpp" -#include +#include "fuse.h" #include + namespace l { static diff --git a/src/fuse_fchown.hpp b/src/fuse_fchown.hpp index 6b043282..991734f6 100644 --- a/src/fuse_fchown.hpp +++ b/src/fuse_fchown.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_fgetattr.cpp b/src/fuse_fgetattr.cpp index eb5242ec..bc8b44d4 100644 --- a/src/fuse_fgetattr.cpp +++ b/src/fuse_fgetattr.cpp @@ -20,7 +20,8 @@ #include "fs_fstat.hpp" #include "fs_inode.hpp" -#include +#include "fuse.h" + namespace l { @@ -50,15 +51,15 @@ namespace FUSE fuse_timeouts_t *timeout_) { int rv; - const Config &config = Config::ro(); + Config::Read cfg; FileInfo *fi = reinterpret_cast(ffi_->fh); rv = l::fgetattr(fi->fd,fi->fusepath,st_); timeout_->entry = ((rv >= 0) ? - config.cache_entry : - config.cache_negative_entry); - timeout_->attr = config.cache_attr; + cfg->cache_entry : + cfg->cache_negative_entry); + timeout_->attr = cfg->cache_attr; return rv; } diff --git a/src/fuse_fgetattr.hpp b/src/fuse_fgetattr.hpp index ce032ac5..e58a061a 100644 --- a/src/fuse_fgetattr.hpp +++ b/src/fuse_fgetattr.hpp @@ -16,12 +16,13 @@ #pragma once -#include +#include "fuse.h" #include #include #include + namespace FUSE { int diff --git a/src/fuse_flock.cpp b/src/fuse_flock.cpp index f1df49fa..4a6a2bb3 100644 --- a/src/fuse_flock.cpp +++ b/src/fuse_flock.cpp @@ -18,7 +18,8 @@ #include "fileinfo.hpp" #include "fs_flock.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_flock.hpp b/src/fuse_flock.hpp index cab3c845..194da4de 100644 --- a/src/fuse_flock.hpp +++ b/src/fuse_flock.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_flush.cpp b/src/fuse_flush.cpp index 33935530..37dc87b2 100644 --- a/src/fuse_flush.cpp +++ b/src/fuse_flush.cpp @@ -19,7 +19,8 @@ #include "fs_close.hpp" #include "fs_dup.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_flush.hpp b/src/fuse_flush.hpp index 27ca708b..b8d447c3 100644 --- a/src/fuse_flush.hpp +++ b/src/fuse_flush.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_free_hide.cpp b/src/fuse_free_hide.cpp index 778d4bf7..4c057d5f 100644 --- a/src/fuse_free_hide.cpp +++ b/src/fuse_free_hide.cpp @@ -19,7 +19,8 @@ #include "fileinfo.hpp" #include "fs_close.hpp" -#include +#include + namespace FUSE { diff --git a/src/fuse_free_hide.hpp b/src/fuse_free_hide.hpp index c8f7c471..8b6be9ee 100644 --- a/src/fuse_free_hide.hpp +++ b/src/fuse_free_hide.hpp @@ -18,7 +18,8 @@ #pragma once -#include +#include + namespace FUSE { diff --git a/src/fuse_fsync.cpp b/src/fuse_fsync.cpp index 96dbc083..1ab9824b 100644 --- a/src/fuse_fsync.cpp +++ b/src/fuse_fsync.cpp @@ -19,11 +19,12 @@ #include "fs_fdatasync.hpp" #include "fs_fsync.hpp" -#include +#include "fuse.h" #include #include + namespace l { static diff --git a/src/fuse_fsync.hpp b/src/fuse_fsync.hpp index c97e569f..58302504 100644 --- a/src/fuse_fsync.hpp +++ b/src/fuse_fsync.hpp @@ -16,12 +16,13 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { int fsync(const fuse_file_info_t *ffi, int isdatasync); - + } diff --git a/src/fuse_fsyncdir.cpp b/src/fuse_fsyncdir.cpp index 21ae8efd..1388b410 100644 --- a/src/fuse_fsyncdir.cpp +++ b/src/fuse_fsyncdir.cpp @@ -18,11 +18,12 @@ #include "dirinfo.hpp" #include "fs_fsync.hpp" -#include +#include "fuse.h" #include #include + namespace l { static diff --git a/src/fuse_fsyncdir.hpp b/src/fuse_fsyncdir.hpp index 7db828cd..e1eba800 100644 --- a/src/fuse_fsyncdir.hpp +++ b/src/fuse_fsyncdir.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_ftruncate.cpp b/src/fuse_ftruncate.cpp index c68a312f..4e3116d4 100644 --- a/src/fuse_ftruncate.cpp +++ b/src/fuse_ftruncate.cpp @@ -18,7 +18,8 @@ #include "fileinfo.hpp" #include "fs_ftruncate.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_ftruncate.hpp b/src/fuse_ftruncate.hpp index 78c8fb9e..16d1c720 100644 --- a/src/fuse_ftruncate.hpp +++ b/src/fuse_ftruncate.hpp @@ -16,11 +16,12 @@ #pragma once -#include +#include "fuse.h" #include #include + namespace FUSE { int diff --git a/src/fuse_futimens.cpp b/src/fuse_futimens.cpp index 2f21b319..8dd27323 100644 --- a/src/fuse_futimens.cpp +++ b/src/fuse_futimens.cpp @@ -18,10 +18,11 @@ #include "fileinfo.hpp" #include "fs_futimens.hpp" -#include +#include "fuse.h" #include + namespace l { static diff --git a/src/fuse_futimens.hpp b/src/fuse_futimens.hpp index 6d149ed7..7e373cc4 100644 --- a/src/fuse_futimens.hpp +++ b/src/fuse_futimens.hpp @@ -16,10 +16,11 @@ #pragma once -#include +#include "fuse.h" #include + namespace FUSE { int diff --git a/src/fuse_getattr.cpp b/src/fuse_getattr.cpp index 54669a09..38ca49ae 100644 --- a/src/fuse_getattr.cpp +++ b/src/fuse_getattr.cpp @@ -22,13 +22,12 @@ #include "symlinkify.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include -#include using std::string; -using std::vector; + namespace l { @@ -59,7 +58,7 @@ namespace l static int - getattr(Policy::Func::Search searchFunc_, + getattr(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, struct stat *st_, @@ -68,7 +67,7 @@ namespace l { int rv; string fullpath; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -87,36 +86,43 @@ namespace l return 0; } -} -namespace FUSE -{ int getattr(const char *fusepath_, struct stat *st_, fuse_timeouts_t *timeout_) { int rv; - const Config &config = Config::ro(); - - if(fusepath_ == config.controlfile) - return l::getattr_controlfile(st_); - + Config::Read cfg; const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - rv = l::getattr(config.func.getattr.policy, - config.branches, + rv = l::getattr(cfg->func.getattr.policy, + cfg->branches, fusepath_, st_, - config.symlinkify, - config.symlinkify_timeout); + cfg->symlinkify, + cfg->symlinkify_timeout); timeout_->entry = ((rv >= 0) ? - config.cache_entry : - config.cache_negative_entry); - timeout_->attr = config.cache_attr; + cfg->cache_entry : + cfg->cache_negative_entry); + timeout_->attr = cfg->cache_attr; return rv; } } + +namespace FUSE +{ + int + getattr(const char *fusepath_, + struct stat *st_, + fuse_timeouts_t *timeout_) + { + if(fusepath_ == CONTROLFILE) + return l::getattr_controlfile(st_); + + return l::getattr(fusepath_,st_,timeout_); + } +} diff --git a/src/fuse_getattr.hpp b/src/fuse_getattr.hpp index a41eccd1..03e4b6c1 100644 --- a/src/fuse_getattr.hpp +++ b/src/fuse_getattr.hpp @@ -20,6 +20,7 @@ #include #include + namespace FUSE { int diff --git a/src/fuse_getxattr.cpp b/src/fuse_getxattr.cpp index e0fd6ca8..2f8145a2 100644 --- a/src/fuse_getxattr.cpp +++ b/src/fuse_getxattr.cpp @@ -24,12 +24,11 @@ #include "ugid.hpp" #include "version.hpp" -#include +#include "fuse.h" #include #include #include -#include #include #include @@ -37,7 +36,7 @@ static const char SECURITY_CAPABILITY[] = "security.capability"; using std::string; -using std::vector; + namespace l { @@ -64,7 +63,7 @@ namespace l static int - getxattr_controlfile(const Config &config_, + getxattr_controlfile(Config::Read &cfg_, const char *attrname_, char *buf_, const size_t count_) @@ -73,13 +72,13 @@ namespace l size_t len; string key; string val; - vector attr; + StrVec attr; if(!str::startswith(attrname_,"user.mergerfs.")) return -ENOATTR; key = &attrname_[14]; - rv = config_.get(key,&val); + rv = cfg_->get(key,&val); if(rv < 0) return rv; @@ -117,16 +116,16 @@ namespace l static int - getxattr_user_mergerfs_allpaths(const Branches &branches_, - const char *fusepath_, - char *buf_, - const size_t count_) + getxattr_user_mergerfs_allpaths(const Branches::CPtr &branches_, + const char *fusepath_, + char *buf_, + const size_t count_) { string concated; - vector paths; - vector branches; + StrVec paths; + StrVec branches; - branches_.to_paths(branches); + branches_->to_paths(branches); fs::findallfiles(branches,fusepath_,&paths); @@ -145,7 +144,7 @@ namespace l char *buf_, const size_t count_) { - vector attr; + StrVec attr; str::split(attrname_,'.',&attr); @@ -163,7 +162,7 @@ namespace l static int - getxattr(Policy::Func::Search searchFunc_, + getxattr(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const char *attrname_, @@ -172,7 +171,7 @@ namespace l { int rv; string fullpath; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -201,26 +200,26 @@ namespace FUSE char *buf_, size_t count_) { - const Config &config = Config::ro(); + Config::Read cfg; - if(fusepath_ == config.controlfile) - return l::getxattr_controlfile(config, + if(fusepath_ == CONTROLFILE) + return l::getxattr_controlfile(cfg, attrname_, buf_, count_); - if((config.security_capability == false) && + if((cfg->security_capability == false) && l::is_attrname_security_capability(attrname_)) return -ENOATTR; - if(config.xattr.to_int()) - return -config.xattr.to_int(); + if(cfg->xattr.to_int()) + return -cfg->xattr.to_int(); const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::getxattr(config.func.getxattr.policy, - config.branches, + return l::getxattr(cfg->func.getxattr.policy, + cfg->branches, fusepath_, attrname_, buf_, diff --git a/src/fuse_getxattr.hpp b/src/fuse_getxattr.hpp index 4ff565c0..60869c87 100644 --- a/src/fuse_getxattr.hpp +++ b/src/fuse_getxattr.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_init.cpp b/src/fuse_init.cpp index adda4c32..2dccd9c8 100644 --- a/src/fuse_init.cpp +++ b/src/fuse_init.cpp @@ -17,7 +17,8 @@ #include "config.hpp" #include "ugid.hpp" -#include +#include "fuse.h" + namespace l { @@ -64,16 +65,16 @@ namespace l static void want_if_capable_max_pages(fuse_conn_info *conn_, - Config &c_) + Config::Write &cfg_) { if(l::capable(conn_,FUSE_CAP_MAX_PAGES)) { l::want(conn_,FUSE_CAP_MAX_PAGES); - conn_->max_pages = c_.fuse_msg_size; + conn_->max_pages = cfg_->fuse_msg_size; } else { - c_.fuse_msg_size = FUSE_DEFAULT_MAX_PAGES_PER_REQ; + cfg_->fuse_msg_size = FUSE_DEFAULT_MAX_PAGES_PER_REQ; } } } @@ -83,24 +84,24 @@ namespace FUSE void * init(fuse_conn_info *conn_) { - Config &config = Config::rw(); + Config::Write cfg; ugid::init(); l::want_if_capable(conn_,FUSE_CAP_ASYNC_DIO); - l::want_if_capable(conn_,FUSE_CAP_ASYNC_READ,&config.async_read); + l::want_if_capable(conn_,FUSE_CAP_ASYNC_READ,&cfg->async_read); l::want_if_capable(conn_,FUSE_CAP_ATOMIC_O_TRUNC); l::want_if_capable(conn_,FUSE_CAP_BIG_WRITES); - l::want_if_capable(conn_,FUSE_CAP_CACHE_SYMLINKS,&config.cache_symlinks); + l::want_if_capable(conn_,FUSE_CAP_CACHE_SYMLINKS,&cfg->cache_symlinks); l::want_if_capable(conn_,FUSE_CAP_DONT_MASK); l::want_if_capable(conn_,FUSE_CAP_IOCTL_DIR); l::want_if_capable(conn_,FUSE_CAP_PARALLEL_DIROPS); - l::want_if_capable(conn_,FUSE_CAP_READDIR_PLUS,&config.readdirplus); + l::want_if_capable(conn_,FUSE_CAP_READDIR_PLUS,&cfg->readdirplus); //l::want_if_capable(conn_,FUSE_CAP_READDIR_PLUS_AUTO); - l::want_if_capable(conn_,FUSE_CAP_POSIX_ACL,&config.posix_acl); - l::want_if_capable(conn_,FUSE_CAP_WRITEBACK_CACHE,&config.writeback_cache); - l::want_if_capable_max_pages(conn_,config); + l::want_if_capable(conn_,FUSE_CAP_POSIX_ACL,&cfg->posix_acl); + l::want_if_capable(conn_,FUSE_CAP_WRITEBACK_CACHE,&cfg->writeback_cache); + l::want_if_capable_max_pages(conn_,cfg); - return &config; + return NULL; } } diff --git a/src/fuse_init.hpp b/src/fuse_init.hpp index d36b9070..5867d646 100644 --- a/src/fuse_init.hpp +++ b/src/fuse_init.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_ioctl.cpp b/src/fuse_ioctl.cpp index 88d49610..db7199e0 100644 --- a/src/fuse_ioctl.cpp +++ b/src/fuse_ioctl.cpp @@ -27,7 +27,7 @@ #include "str.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -134,7 +134,7 @@ namespace l static int - ioctl_dir_base(Policy::Func::Search searchFunc_, + ioctl_dir_base(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const uint32_t cmd_, @@ -144,7 +144,7 @@ namespace l int fd; int rv; string fullpath; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -170,13 +170,13 @@ namespace l void *data_, uint32_t *out_bufsz_) { - DirInfo *di = reinterpret_cast(ffi_->fh); - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + DirInfo *di = reinterpret_cast(ffi_->fh); + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::ioctl_dir_base(config.func.open.policy, - config.branches, + return l::ioctl_dir_base(cfg->func.open.policy, + cfg->branches, di->fusepath.c_str(), cmd_, data_, @@ -203,10 +203,10 @@ namespace l int read_keys(void *data_) { - std::string keys; - const Config &config = Config::ro(); + Config::Read cfg; + std::string keys; - config.keys(keys); + cfg->keys(keys); return l::strcpy(keys,data_); } @@ -215,17 +215,17 @@ namespace l int read_val(void *data_) { + Config::Read cfg; int rv; char *data; std::string key; std::string val; - const Config &config = Config::ro(); data = (char*)data_; data[sizeof(IOCTL_BUF) - 1] = '\0'; key = data; - rv = config.get(key,&val); + rv = cfg->get(key,&val); if(rv < 0) return rv; @@ -236,11 +236,11 @@ namespace l int write_val(void *data_) { + Config::Write cfg; char *data; std::string kv; std::string key; std::string val; - Config &config = Config::rw(); data = (char*)data_; data[sizeof(IOCTL_BUF) - 1] = '\0'; @@ -248,18 +248,18 @@ namespace l kv = data; str::splitkv(kv,'=',&key,&val); - return config.set(key,val); + return cfg->set(key,val); } static int - file_basepath(Policy::Func::Search searchFunc_, + file_basepath(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, void *data_) { int rv; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -273,11 +273,11 @@ namespace l file_basepath(const fuse_file_info_t *ffi_, void *data_) { - const Config &config = Config::ro(); + Config::Read cfg; std::string &fusepath = reinterpret_cast(ffi_->fh)->fusepath; - return l::file_basepath(config.func.open.policy, - config.branches, + return l::file_basepath(cfg->func.open.policy, + cfg->branches, fusepath.c_str(), data_); } @@ -294,14 +294,14 @@ namespace l static int - file_fullpath(Policy::Func::Search searchFunc_, + file_fullpath(const Policy::Search &searchFunc_, const Branches &branches_, const string &fusepath_, void *data_) { int rv; string fullpath; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -317,11 +317,11 @@ namespace l file_fullpath(const fuse_file_info_t *ffi_, void *data_) { - const Config &config = Config::ro(); + Config::Read cfg; std::string &fusepath = reinterpret_cast(ffi_->fh)->fusepath; - return l::file_fullpath(config.func.open.policy, - config.branches, + return l::file_fullpath(cfg->func.open.policy, + cfg->branches, fusepath, data_); } @@ -331,13 +331,13 @@ namespace l file_allpaths(const fuse_file_info_t *ffi_, void *data_) { + Config::Read cfg; string concated; - vector paths; - vector branches; + StrVec paths; + StrVec branches; string &fusepath = reinterpret_cast(ffi_->fh)->fusepath; - const Config &config = Config::ro(); - config.branches.to_paths(branches); + cfg->branches->to_paths(branches); fs::findallfiles(branches,fusepath.c_str(),&paths); diff --git a/src/fuse_ioctl.hpp b/src/fuse_ioctl.hpp index 98b6f863..11ea6233 100644 --- a/src/fuse_ioctl.hpp +++ b/src/fuse_ioctl.hpp @@ -16,7 +16,10 @@ #pragma once -#include +#include + +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_link.cpp b/src/fuse_link.cpp index ecc1ed41..4b06adcf 100644 --- a/src/fuse_link.cpp +++ b/src/fuse_link.cpp @@ -21,7 +21,7 @@ #include "fs_path.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -29,6 +29,7 @@ using std::string; using std::vector; + namespace error { static @@ -73,11 +74,11 @@ namespace l static int - link_create_path_loop(const vector &oldbasepaths_, - const string &newbasepath_, - const char *oldfusepath_, - const char *newfusepath_, - const string &newfusedirpath_) + link_create_path_loop(const StrVec &oldbasepaths_, + const string &newbasepath_, + const char *oldfusepath_, + const char *newfusepath_, + const string &newfusedirpath_) { int rv; int error; @@ -99,16 +100,16 @@ namespace l static int - link_create_path(Policy::Func::Search searchFunc_, - Policy::Func::Action actionFunc_, + link_create_path(const Policy::Search &searchFunc_, + const Policy::Action &actionFunc_, const Branches &branches_, const char *oldfusepath_, const char *newfusepath_) { int rv; string newfusedirpath; - vector oldbasepaths; - vector newbasepaths; + StrVec oldbasepaths; + StrVec newbasepaths; rv = actionFunc_(branches_,oldfusepath_,&oldbasepaths); if(rv == -1) @@ -127,8 +128,8 @@ namespace l static int - clonepath_if_would_create(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + clonepath_if_would_create(const Policy::Search &searchFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const string &oldbasepath_, const char *oldfusepath_, @@ -136,7 +137,7 @@ namespace l { int rv; string newfusedirpath; - vector newbasepath; + StrVec newbasepath; newfusedirpath = fs::path::dirname(newfusepath_); @@ -156,8 +157,8 @@ namespace l static int - link_preserve_path_core(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + link_preserve_path_core(const Policy::Search &searchFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const string &oldbasepath_, const char *oldfusepath_, @@ -187,12 +188,12 @@ namespace l static int - link_preserve_path_loop(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + link_preserve_path_loop(const Policy::Search &searchFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const char *oldfusepath_, const char *newfusepath_, - const vector &oldbasepaths_) + const StrVec &oldbasepaths_) { int error; @@ -211,15 +212,15 @@ namespace l static int - link_preserve_path(Policy::Func::Search searchFunc_, - Policy::Func::Action actionFunc_, - Policy::Func::Create createFunc_, + link_preserve_path(const Policy::Search &searchFunc_, + const Policy::Action &actionFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const char *oldfusepath_, const char *newfusepath_) { int rv; - vector oldbasepaths; + StrVec oldbasepaths; rv = actionFunc_(branches_,oldfusepath_,&oldbasepaths); if(rv == -1) @@ -238,21 +239,21 @@ namespace FUSE link(const char *from_, const char *to_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - if(config.func.create.policy->path_preserving() && !config.ignorepponrename) - return l::link_preserve_path(config.func.getattr.policy, - config.func.link.policy, - config.func.create.policy, - config.branches, + if(cfg->func.create.policy.path_preserving() && !cfg->ignorepponrename) + return l::link_preserve_path(cfg->func.getattr.policy, + cfg->func.link.policy, + cfg->func.create.policy, + cfg->branches, from_, to_); - return l::link_create_path(config.func.link.policy, - config.func.create.policy, - config.branches, + return l::link_create_path(cfg->func.getattr.policy, + cfg->func.link.policy, + cfg->branches, from_, to_); } diff --git a/src/fuse_link.hpp b/src/fuse_link.hpp index bb1eb2ec..ab601f88 100644 --- a/src/fuse_link.hpp +++ b/src/fuse_link.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_listxattr.cpp b/src/fuse_listxattr.cpp index 29b3f57c..1f69b15c 100644 --- a/src/fuse_listxattr.cpp +++ b/src/fuse_listxattr.cpp @@ -14,7 +14,6 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include "buildvector.hpp" #include "category.hpp" #include "config.hpp" #include "errno.hpp" @@ -23,27 +22,26 @@ #include "ugid.hpp" #include "xattr.hpp" -#include +#include "fuse.h" #include -#include #include using std::string; -using std::vector; + namespace l { static int - listxattr_controlfile(const Config &config_, + listxattr_controlfile(Config::Read &cfg_, char *list_, const size_t size_) { string xattrs; - config_.keys_xattr(xattrs); + cfg_->keys_xattr(xattrs); if(size_ == 0) return xattrs.size(); @@ -57,7 +55,7 @@ namespace l static int - listxattr(Policy::Func::Search searchFunc_, + listxattr(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, char *list_, @@ -65,7 +63,7 @@ namespace l { int rv; string fullpath; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -86,12 +84,12 @@ namespace FUSE char *list_, size_t size_) { - const Config &config = Config::ro(); + Config::Read cfg; - if(fusepath_ == config.controlfile) - return l::listxattr_controlfile(config,list_,size_); + if(fusepath_ == CONTROLFILE) + return l::listxattr_controlfile(cfg,list_,size_); - switch(config.xattr) + switch(cfg->xattr) { case XAttr::ENUM::PASSTHROUGH: break; @@ -104,8 +102,8 @@ namespace FUSE const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::listxattr(config.func.listxattr.policy, - config.branches, + return l::listxattr(cfg->func.listxattr.policy, + cfg->branches, fusepath_, list_, size_); diff --git a/src/fuse_listxattr.hpp b/src/fuse_listxattr.hpp index cef0b91c..f06c32c8 100644 --- a/src/fuse_listxattr.hpp +++ b/src/fuse_listxattr.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_mkdir.cpp b/src/fuse_mkdir.cpp index 66efccd7..c8bc3e57 100644 --- a/src/fuse_mkdir.cpp +++ b/src/fuse_mkdir.cpp @@ -20,15 +20,15 @@ #include "fs_clonepath.hpp" #include "fs_mkdir.hpp" #include "fs_path.hpp" +#include "policy.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include -#include using std::string; -using std::vector; + namespace error { @@ -84,12 +84,12 @@ namespace l static int - mkdir_loop(const string &existingpath_, - const vector &createpaths_, - const char *fusepath_, - const string &fusedirpath_, - const mode_t mode_, - const mode_t umask_) + mkdir_loop(const string &existingpath_, + const StrVec &createpaths_, + const char *fusepath_, + const string &fusedirpath_, + const mode_t mode_, + const mode_t umask_) { int rv; int error; @@ -113,8 +113,8 @@ namespace l static int - mkdir(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + mkdir(const Policy::Search &getattrPolicy_, + const Policy::Create &mkdirPolicy_, const Branches &branches_, const char *fusepath_, const mode_t mode_, @@ -122,16 +122,16 @@ namespace l { int rv; string fusedirpath; - vector createpaths; - vector existingpaths; + StrVec createpaths; + StrVec existingpaths; fusedirpath = fs::path::dirname(fusepath_); - rv = searchFunc_(branches_,fusedirpath,&existingpaths); + rv = getattrPolicy_(branches_,fusedirpath.c_str(),&existingpaths); if(rv == -1) return -errno; - rv = createFunc_(branches_,fusedirpath,&createpaths); + rv = mkdirPolicy_(branches_,fusedirpath.c_str(),&createpaths); if(rv == -1) return -errno; @@ -150,13 +150,13 @@ namespace FUSE mkdir(const char *fusepath_, mode_t mode_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::mkdir(config.func.getattr.policy, - config.func.mkdir.policy, - config.branches, + return l::mkdir(cfg->func.getattr.policy, + cfg->func.mkdir.policy, + cfg->branches, fusepath_, mode_, fc->umask); diff --git a/src/fuse_mkdir.hpp b/src/fuse_mkdir.hpp index e70fa64e..f2c515b2 100644 --- a/src/fuse_mkdir.hpp +++ b/src/fuse_mkdir.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_mknod.cpp b/src/fuse_mknod.cpp index 042efefc..ad7b4f46 100644 --- a/src/fuse_mknod.cpp +++ b/src/fuse_mknod.cpp @@ -22,7 +22,7 @@ #include "fs_path.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -30,6 +30,7 @@ using std::string; using std::vector; + namespace error { static @@ -115,8 +116,8 @@ namespace l static int - mknod(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + mknod(const Policy::Search &searchFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const char *fusepath_, const mode_t mode_, @@ -151,13 +152,13 @@ namespace FUSE mode_t mode_, dev_t rdev_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::mknod(config.func.getattr.policy, - config.func.mknod.policy, - config.branches, + return l::mknod(cfg->func.getattr.policy, + cfg->func.mknod.policy, + cfg->branches, fusepath_, mode_, fc->umask, diff --git a/src/fuse_mknod.hpp b/src/fuse_mknod.hpp index adb56d8d..6350f310 100644 --- a/src/fuse_mknod.hpp +++ b/src/fuse_mknod.hpp @@ -17,6 +17,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_open.cpp b/src/fuse_open.cpp index 583714f4..2e86fe5a 100644 --- a/src/fuse_open.cpp +++ b/src/fuse_open.cpp @@ -23,7 +23,6 @@ #include "fs_open.hpp" #include "fs_path.hpp" #include "fs_stat.hpp" -#include "policy_cache.hpp" #include "stat_util.hpp" #include "ugid.hpp" @@ -35,6 +34,7 @@ using std::string; using std::vector; + namespace l { static @@ -117,15 +117,15 @@ namespace l static void - config_to_ffi_flags(const Config &config_, + config_to_ffi_flags(Config::Read &cfg_, fuse_file_info_t *ffi_) { - switch(config_.cache_files) + switch(cfg_->cache_files) { case CacheFiles::ENUM::LIBFUSE: - ffi_->direct_io = config_.direct_io; - ffi_->keep_cache = config_.kernel_cache; - ffi_->auto_cache = config_.auto_cache; + ffi_->direct_io = cfg_->direct_io; + ffi_->keep_cache = cfg_->kernel_cache; + ffi_->auto_cache = cfg_->auto_cache; break; case CacheFiles::ENUM::OFF: ffi_->direct_io = 1; @@ -180,8 +180,7 @@ namespace l static int - open(Policy::Func::Search searchFunc_, - PolicyCache &cache, + open(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const int flags_, @@ -190,13 +189,13 @@ namespace l uint64_t *fh_) { int rv; - string basepath; + StrVec basepaths; - rv = cache(searchFunc_,branches_,fusepath_,&basepath); + rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; - return l::open_core(basepath,fusepath_,flags_,link_cow_,nfsopenhack_,fh_); + return l::open_core(basepaths[0],fusepath_,flags_,link_cow_,nfsopenhack_,fh_); } } @@ -206,22 +205,21 @@ namespace FUSE open(const char *fusepath_, fuse_file_info_t *ffi_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - l::config_to_ffi_flags(config,ffi_); + l::config_to_ffi_flags(cfg,ffi_); - if(config.writeback_cache) + if(cfg->writeback_cache) l::tweak_flags_writeback_cache(&ffi_->flags); - return l::open(config.func.open.policy, - config.open_cache, - config.branches, + return l::open(cfg->func.open.policy, + cfg->branches, fusepath_, ffi_->flags, - config.link_cow, - config.nfsopenhack, + cfg->link_cow, + cfg->nfsopenhack, &ffi_->fh); } } diff --git a/src/fuse_open.hpp b/src/fuse_open.hpp index 2d7cadee..34c758bc 100644 --- a/src/fuse_open.hpp +++ b/src/fuse_open.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_opendir.cpp b/src/fuse_opendir.cpp index 6bd5e864..43c706ae 100644 --- a/src/fuse_opendir.cpp +++ b/src/fuse_opendir.cpp @@ -17,7 +17,8 @@ #include "config.hpp" #include "dirinfo.hpp" -#include +#include "fuse.h" + namespace FUSE { @@ -25,11 +26,11 @@ namespace FUSE opendir(const char *fusepath_, fuse_file_info_t *ffi_) { - const Config &config = Config::ro(); + Config::Read cfg; ffi_->fh = reinterpret_cast(new DirInfo(fusepath_)); - if(config.cache_readdir) + if(cfg->cache_readdir) { ffi_->keep_cache = 1; ffi_->cache_readdir = 1; diff --git a/src/fuse_opendir.hpp b/src/fuse_opendir.hpp index 7bc83339..1a2dca84 100644 --- a/src/fuse_opendir.hpp +++ b/src/fuse_opendir.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_prepare_hide.cpp b/src/fuse_prepare_hide.cpp index 73c8e74b..fbd741ea 100644 --- a/src/fuse_prepare_hide.cpp +++ b/src/fuse_prepare_hide.cpp @@ -18,10 +18,11 @@ #include "fuse_open.hpp" -#include +#include #include + namespace FUSE { int diff --git a/src/fuse_prepare_hide.hpp b/src/fuse_prepare_hide.hpp index 696f5819..9e38760b 100644 --- a/src/fuse_prepare_hide.hpp +++ b/src/fuse_prepare_hide.hpp @@ -18,7 +18,8 @@ #pragma once -#include +#include + namespace FUSE { diff --git a/src/fuse_read.cpp b/src/fuse_read.cpp index 34a4b2e5..cc177de4 100644 --- a/src/fuse_read.cpp +++ b/src/fuse_read.cpp @@ -18,7 +18,8 @@ #include "fileinfo.hpp" #include "fs_read.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_read.hpp b/src/fuse_read.hpp index 21110ee2..b15c8d27 100644 --- a/src/fuse_read.hpp +++ b/src/fuse_read.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_read_buf.cpp b/src/fuse_read_buf.cpp index d43355a9..733940f1 100644 --- a/src/fuse_read_buf.cpp +++ b/src/fuse_read_buf.cpp @@ -17,13 +17,14 @@ #include "errno.hpp" #include "fileinfo.hpp" -#include +#include "fuse.h" #include #include typedef struct fuse_bufvec fuse_bufvec; + namespace l { static diff --git a/src/fuse_read_buf.hpp b/src/fuse_read_buf.hpp index 34ecc46f..f614e916 100644 --- a/src/fuse_read_buf.hpp +++ b/src/fuse_read_buf.hpp @@ -16,10 +16,11 @@ #pragma once -#include +#include "fuse.h" #include + namespace FUSE { int diff --git a/src/fuse_readdir.cpp b/src/fuse_readdir.cpp index e9607442..2ef0b3e4 100644 --- a/src/fuse_readdir.cpp +++ b/src/fuse_readdir.cpp @@ -24,7 +24,8 @@ #include "rwlock.hpp" #include "ugid.hpp" -#include +#include "fuse.h" + namespace FUSE { @@ -32,18 +33,18 @@ namespace FUSE readdir(const fuse_file_info_t *ffi_, fuse_dirents_t *buf_) { - DirInfo *di = reinterpret_cast(ffi_->fh); - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + DirInfo *di = reinterpret_cast(ffi_->fh); + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - switch(config.readdir) + switch(cfg->readdir) { case ReadDir::ENUM::LINUX: - return FUSE::readdir_linux(config.branches,di->fusepath.c_str(),buf_); + return FUSE::readdir_linux(cfg->branches,di->fusepath.c_str(),buf_); default: case ReadDir::ENUM::POSIX: - return FUSE::readdir_posix(config.branches,di->fusepath.c_str(),buf_); + return FUSE::readdir_posix(cfg->branches,di->fusepath.c_str(),buf_); } } } diff --git a/src/fuse_readdir.hpp b/src/fuse_readdir.hpp index 92741edd..c82937c3 100644 --- a/src/fuse_readdir.hpp +++ b/src/fuse_readdir.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_readdir_linux.cpp b/src/fuse_readdir_linux.cpp index 4d0cf062..7bbf2ece 100644 --- a/src/fuse_readdir_linux.cpp +++ b/src/fuse_readdir_linux.cpp @@ -14,7 +14,7 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include "branch.hpp" +#include "branches.hpp" #include "errno.hpp" #include "fs_close.hpp" #include "fs_devid.hpp" @@ -26,10 +26,9 @@ #include "hashset.hpp" #include "linux_dirent64.h" #include "mempools.hpp" -#include "rwlock.hpp" -#include -#include +#include "fuse.h" +#include "fuse_dirents.h" #include #include @@ -53,9 +52,9 @@ namespace l static int - readdir(const BranchVec &branches_, - const char *dirname_, - fuse_dirents_t *buf_) + readdir(const Branches::CPtr &branches_, + const char *dirname_, + fuse_dirents_t *buf_) { int rv; dev_t dev; @@ -70,20 +69,18 @@ namespace l if(buf == NULL) return -ENOMEM; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { int dirfd; int64_t nread; - basepath = fs::path::make(branches_[i].path,dirname_); + basepath = fs::path::make(branch.path,dirname_); dirfd = fs::open_dir_ro(basepath); if(dirfd == -1) continue; dev = fs::devid(dirfd); - if(dev == (dev_t)-1) - dev = i; for(;;) { @@ -122,25 +119,14 @@ namespace l return 0; } - - static - int - readdir(const Branches &branches_, - const char *dirname_, - fuse_dirents_t *buf_) - { - rwlock::ReadGuard guard(branches_.lock); - - return l::readdir(branches_.vec,dirname_,buf_); - } } namespace FUSE { int - readdir_linux(const Branches &branches_, - const char *dirname_, - fuse_dirents_t *buf_) + readdir_linux(const Branches::CPtr &branches_, + const char *dirname_, + fuse_dirents_t *buf_) { return l::readdir(branches_,dirname_,buf_); } diff --git a/src/fuse_readdir_linux.hpp b/src/fuse_readdir_linux.hpp index d6b9b09d..26cfff6a 100644 --- a/src/fuse_readdir_linux.hpp +++ b/src/fuse_readdir_linux.hpp @@ -18,16 +18,17 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" -#include +#include "fuse.h" + +#include -#include namespace FUSE { int - readdir_linux(const Branches &branches, - const char *dirname, - fuse_dirents_t *buf); + readdir_linux(const Branches::CPtr &branches, + const char *dirname, + fuse_dirents_t *buf); } diff --git a/src/fuse_readdir_plus.cpp b/src/fuse_readdir_plus.cpp index 6ee1f52a..e9870fa2 100644 --- a/src/fuse_readdir_plus.cpp +++ b/src/fuse_readdir_plus.cpp @@ -24,7 +24,8 @@ #include "rwlock.hpp" #include "ugid.hpp" -#include +#include "fuse.h" + namespace FUSE { @@ -32,25 +33,25 @@ namespace FUSE readdir_plus(const fuse_file_info_t *ffi_, fuse_dirents_t *buf_) { - DirInfo *di = reinterpret_cast(ffi_->fh); - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + DirInfo *di = reinterpret_cast(ffi_->fh); + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - switch(config.readdir) + switch(cfg->readdir) { case ReadDir::ENUM::LINUX: - return FUSE::readdir_plus_linux(config.branches, + return FUSE::readdir_plus_linux(cfg->branches, di->fusepath.c_str(), - config.cache_entry, - config.cache_attr, + cfg->cache_entry, + cfg->cache_attr, buf_); default: case ReadDir::ENUM::POSIX: - return FUSE::readdir_plus_posix(config.branches, + return FUSE::readdir_plus_posix(cfg->branches, di->fusepath.c_str(), - config.cache_entry, - config.cache_attr, + cfg->cache_entry, + cfg->cache_attr, buf_); } } diff --git a/src/fuse_readdir_plus.hpp b/src/fuse_readdir_plus.hpp index 20278f0c..8e298e61 100644 --- a/src/fuse_readdir_plus.hpp +++ b/src/fuse_readdir_plus.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_readdir_plus_linux.cpp b/src/fuse_readdir_plus_linux.cpp index ad643a5f..9e87568b 100644 --- a/src/fuse_readdir_plus_linux.cpp +++ b/src/fuse_readdir_plus_linux.cpp @@ -14,7 +14,7 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include "branch.hpp" +#include "branches.hpp" #include "errno.hpp" #include "fs_close.hpp" #include "fs_devid.hpp" @@ -28,8 +28,8 @@ #include "linux_dirent64.h" #include "mempools.hpp" -#include -#include +#include "fuse.h" +#include "fuse_dirents.h" #include #include @@ -53,11 +53,11 @@ namespace l static int - readdir_plus(const BranchVec &branches_, - const char *dirname_, - const uint64_t entry_timeout_, - const uint64_t attr_timeout_, - fuse_dirents_t *buf_) + readdir_plus(const Branches::CPtr &branches_, + const char *dirname_, + const uint64_t entry_timeout_, + const uint64_t attr_timeout_, + fuse_dirents_t *buf_) { int rv; dev_t dev; @@ -78,20 +78,17 @@ namespace l entry.attr_valid = attr_timeout_; entry.entry_valid_nsec = 0; entry.attr_valid_nsec = 0; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { int dirfd; int64_t nread; - basepath = fs::path::make(branches_[i].path,dirname_); + basepath = fs::path::make(branch.path,dirname_); dirfd = fs::open_dir_ro(basepath); if(dirfd == -1) continue; - dev = fs::devid(dirfd); - if(dev == (dev_t)-1) - dev = i; for(;;) { @@ -136,29 +133,16 @@ namespace l return 0; } - - static - int - readdir_plus(const Branches &branches_, - const char *dirname_, - const uint64_t entry_timeout_, - const uint64_t attr_timeout_, - fuse_dirents_t *buf_) - { - rwlock::ReadGuard guard(branches_.lock); - - return l::readdir_plus(branches_.vec,dirname_,entry_timeout_,attr_timeout_,buf_); - } } namespace FUSE { int - readdir_plus_linux(const Branches &branches_, - const char *dirname_, - const uint64_t entry_timeout_, - const uint64_t attr_timeout_, - fuse_dirents_t *buf_) + readdir_plus_linux(const Branches::CPtr &branches_, + const char *dirname_, + const uint64_t entry_timeout_, + const uint64_t attr_timeout_, + fuse_dirents_t *buf_) { return l::readdir_plus(branches_,dirname_,entry_timeout_,attr_timeout_,buf_); } diff --git a/src/fuse_readdir_plus_linux.hpp b/src/fuse_readdir_plus_linux.hpp index 68020b7f..a0bca539 100644 --- a/src/fuse_readdir_plus_linux.hpp +++ b/src/fuse_readdir_plus_linux.hpp @@ -18,18 +18,19 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" -#include +#include "fuse.h" + +#include -#include namespace FUSE { int - readdir_plus_linux(const Branches &branches, - const char *dirname, - const uint64_t entry_timeout, - const uint64_t attr_timeout, - fuse_dirents_t *buf); + readdir_plus_linux(const Branches::CPtr &branches, + const char *dirname, + const uint64_t entry_timeout, + const uint64_t attr_timeout, + fuse_dirents_t *buf); } diff --git a/src/fuse_readdir_plus_posix.cpp b/src/fuse_readdir_plus_posix.cpp index 6f809684..b78f9516 100644 --- a/src/fuse_readdir_plus_posix.cpp +++ b/src/fuse_readdir_plus_posix.cpp @@ -16,7 +16,7 @@ #define _DEFAULT_SOURCE -#include "branch.hpp" +#include "branches.hpp" #include "errno.hpp" #include "fs_closedir.hpp" #include "fs_devid.hpp" @@ -28,10 +28,9 @@ #include "fs_readdir.hpp" #include "fs_stat.hpp" #include "hashset.hpp" -#include "rwlock.hpp" -#include -#include +#include "fuse.h" +#include "fuse_dirents.h" #include #include @@ -58,11 +57,11 @@ namespace l static int - readdir_plus(const BranchVec &branches_, - const char *dirname_, - const uint64_t entry_timeout_, - const uint64_t attr_timeout_, - fuse_dirents_t *buf_) + readdir_plus(const Branches::CPtr &branches_, + const char *dirname_, + const uint64_t entry_timeout_, + const uint64_t attr_timeout_, + fuse_dirents_t *buf_) { dev_t dev; HashSet names; @@ -78,13 +77,13 @@ namespace l entry.attr_valid = attr_timeout_; entry.entry_valid_nsec = 0; entry.attr_valid_nsec = 0; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { int rv; int dirfd; DIR *dh; - basepath = fs::path::make(branches_[i].path,dirname_); + basepath = fs::path::make(branch.path,dirname_); dh = fs::opendir(basepath); if(!dh) @@ -92,8 +91,6 @@ namespace l dirfd = fs::dirfd(dh); dev = fs::devid(dirfd); - if(dev == (dev_t)-1) - dev = i; rv = 0; for(struct dirent *de = fs::readdir(dh); de && !rv; de = fs::readdir(dh)) @@ -127,33 +124,16 @@ namespace l return 0; } - - static - int - readdir_plus(const Branches &branches_, - const char *dirname_, - const uint64_t entry_timeout_, - const uint64_t attr_timeout_, - fuse_dirents_t *buf_) - { - rwlock::ReadGuard guard(branches_.lock); - - return l::readdir_plus(branches_.vec, - dirname_, - entry_timeout_, - attr_timeout_, - buf_); - } } namespace FUSE { int - readdir_plus_posix(const Branches &branches_, - const char *dirname_, - const uint64_t entry_timeout_, - const uint64_t attr_timeout_, - fuse_dirents_t *buf_) + readdir_plus_posix(const Branches::CPtr &branches_, + const char *dirname_, + const uint64_t entry_timeout_, + const uint64_t attr_timeout_, + fuse_dirents_t *buf_) { return l::readdir_plus(branches_,dirname_,entry_timeout_,attr_timeout_,buf_); } diff --git a/src/fuse_readdir_plus_posix.hpp b/src/fuse_readdir_plus_posix.hpp index ac040673..910cab5f 100644 --- a/src/fuse_readdir_plus_posix.hpp +++ b/src/fuse_readdir_plus_posix.hpp @@ -18,18 +18,19 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" -#include +#include "fuse.h" + +#include -#include namespace FUSE { int - readdir_plus_posix(const Branches &branches, - const char *dirname, - const uint64_t entry_timeout, - const uint64_t attr_timeout, - fuse_dirents_t *buf); + readdir_plus_posix(const Branches::CPtr &branches, + const char *dirname, + const uint64_t entry_timeout, + const uint64_t attr_timeout, + fuse_dirents_t *buf); } diff --git a/src/fuse_readdir_posix.cpp b/src/fuse_readdir_posix.cpp index ba944a83..5d3bf961 100644 --- a/src/fuse_readdir_posix.cpp +++ b/src/fuse_readdir_posix.cpp @@ -16,7 +16,7 @@ #define _DEFAULT_SOURCE -#include "branch.hpp" +#include "branches.hpp" #include "errno.hpp" #include "fs_closedir.hpp" #include "fs_devid.hpp" @@ -27,10 +27,9 @@ #include "fs_readdir.hpp" #include "fs_stat.hpp" #include "hashset.hpp" -#include "rwlock.hpp" -#include -#include +#include "fuse.h" +#include "fuse_dirents.h" #include #include @@ -40,6 +39,7 @@ using std::string; using std::vector; + namespace l { static @@ -57,9 +57,9 @@ namespace l static int - readdir(const BranchVec &branches_, - const char *dirname_, - fuse_dirents_t *buf_) + readdir(const Branches::CPtr &branches_, + const char *dirname_, + fuse_dirents_t *buf_) { dev_t dev; HashSet names; @@ -67,13 +67,13 @@ namespace l string fullpath; uint64_t namelen; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { int rv; int dirfd; DIR *dh; - basepath = fs::path::make(branches_[i].path,dirname_); + basepath = fs::path::make(branch.path,dirname_); dh = fs::opendir(basepath); if(!dh) @@ -81,8 +81,6 @@ namespace l dirfd = fs::dirfd(dh); dev = fs::devid(dirfd); - if(dev == (dev_t)-1) - dev = i; rv = 0; for(struct dirent *de = fs::readdir(dh); de && !rv; de = fs::readdir(dh)) @@ -110,25 +108,14 @@ namespace l return 0; } - - static - int - readdir(const Branches &branches_, - const char *dirname_, - fuse_dirents_t *buf_) - { - rwlock::ReadGuard guard(branches_.lock); - - return l::readdir(branches_.vec,dirname_,buf_); - } } namespace FUSE { int - readdir_posix(const Branches &branches_, - const char *dirname_, - fuse_dirents_t *buf_) + readdir_posix(const Branches::CPtr &branches_, + const char *dirname_, + fuse_dirents_t *buf_) { return l::readdir(branches_,dirname_,buf_); } diff --git a/src/fuse_readdir_posix.hpp b/src/fuse_readdir_posix.hpp index 04eea9d3..36d08dbb 100644 --- a/src/fuse_readdir_posix.hpp +++ b/src/fuse_readdir_posix.hpp @@ -18,16 +18,17 @@ #pragma once -#include "branch.hpp" +#include "branches.hpp" -#include +#include "fuse.h" + +#include -#include namespace FUSE { int - readdir_posix(const Branches &branches, - const char *dirname, - fuse_dirents_t *buf); + readdir_posix(const Branches::CPtr &branches, + const char *dirname, + fuse_dirents_t *buf); } diff --git a/src/fuse_readlink.cpp b/src/fuse_readlink.cpp index 8cebea19..e24354f4 100644 --- a/src/fuse_readlink.cpp +++ b/src/fuse_readlink.cpp @@ -22,12 +22,12 @@ #include "symlinkify.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include using std::string; -using std::vector; + namespace l { @@ -92,7 +92,7 @@ namespace l static int - readlink(Policy::Func::Search searchFunc_, + readlink(const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, char *buf_, @@ -101,7 +101,7 @@ namespace l const time_t symlinkify_timeout_) { int rv; - vector basepaths; + StrVec basepaths; rv = searchFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -119,16 +119,16 @@ namespace FUSE char *buf_, size_t size_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::readlink(config.func.readlink.policy, - config.branches, + return l::readlink(cfg->func.readlink.policy, + cfg->branches, fusepath_, buf_, size_, - config.symlinkify, - config.symlinkify_timeout); + cfg->symlinkify, + cfg->symlinkify_timeout); } } diff --git a/src/fuse_readlink.hpp b/src/fuse_readlink.hpp index e9f37a15..b7e5e6f2 100644 --- a/src/fuse_readlink.hpp +++ b/src/fuse_readlink.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_release.cpp b/src/fuse_release.cpp index 88c1357e..03b6dcdb 100644 --- a/src/fuse_release.cpp +++ b/src/fuse_release.cpp @@ -20,10 +20,11 @@ #include "fs_close.hpp" #include "fs_fadvise.hpp" -#include +#include "fuse.h" #include + namespace l { static @@ -52,12 +53,9 @@ namespace FUSE int release(const fuse_file_info_t *ffi_) { - const Config &config = Config::ro(); + Config::Read cfg; FileInfo *fi = reinterpret_cast(ffi_->fh); - if(config.open_cache.timeout) - config.open_cache.cleanup(10); - - return l::release(fi,config.dropcacheonclose); + return l::release(fi,cfg->dropcacheonclose); } } diff --git a/src/fuse_release.hpp b/src/fuse_release.hpp index 547490fa..94858839 100644 --- a/src/fuse_release.hpp +++ b/src/fuse_release.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_releasedir.cpp b/src/fuse_releasedir.cpp index b4a096ac..8a7c15c5 100644 --- a/src/fuse_releasedir.cpp +++ b/src/fuse_releasedir.cpp @@ -17,7 +17,8 @@ #include "config.hpp" #include "dirinfo.hpp" -#include +#include "fuse.h" + namespace l { diff --git a/src/fuse_releasedir.hpp b/src/fuse_releasedir.hpp index caf7acce..28d278e0 100644 --- a/src/fuse_releasedir.hpp +++ b/src/fuse_releasedir.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_removexattr.cpp b/src/fuse_removexattr.cpp index 9f0ba7ef..d22367ac 100644 --- a/src/fuse_removexattr.cpp +++ b/src/fuse_removexattr.cpp @@ -21,7 +21,7 @@ #include "policy_rv.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -84,8 +84,8 @@ namespace l static int - removexattr(Policy::Func::Action actionFunc_, - Policy::Func::Search searchFunc_, + removexattr(const Policy::Action &actionFunc_, + const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const char *attrname_) @@ -119,19 +119,20 @@ namespace FUSE removexattr(const char *fusepath_, const char *attrname_) { - const Config &config = Config::ro(); + Config::Read cfg; - if(fusepath_ == config.controlfile) + if(fusepath_ == CONTROLFILE) return -ENOATTR; - if(config.xattr.to_int()) - return -config.xattr.to_int(); + + if(cfg->xattr.to_int()) + return -cfg->xattr.to_int(); const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::removexattr(config.func.removexattr.policy, - config.func.getxattr.policy, - config.branches, + return l::removexattr(cfg->func.removexattr.policy, + cfg->func.getxattr.policy, + cfg->branches, fusepath_, attrname_); } diff --git a/src/fuse_removexattr.hpp b/src/fuse_removexattr.hpp index 7dbef5de..4c7357f9 100644 --- a/src/fuse_removexattr.hpp +++ b/src/fuse_removexattr.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_rename.cpp b/src/fuse_rename.cpp index e9fdf0cf..541bce8e 100644 --- a/src/fuse_rename.cpp +++ b/src/fuse_rename.cpp @@ -24,10 +24,8 @@ #include #include -#include using std::string; -using std::vector; namespace error @@ -52,8 +50,8 @@ namespace error static bool -member(const vector &haystack, - const string &needle) +member(const StrVec &haystack, + const string &needle) { for(size_t i = 0, ei = haystack.size(); i != ei; i++) { @@ -66,7 +64,7 @@ member(const vector &haystack, static void -_remove(const vector &toremove) +_remove(const StrVec &toremove) { for(size_t i = 0, ei = toremove.size(); i != ei; i++) fs::remove(toremove[i]); @@ -74,14 +72,14 @@ _remove(const vector &toremove) static void -_rename_create_path_core(const vector &oldbasepaths, - const string &oldbasepath, - const string &newbasepath, - const char *oldfusepath, - const char *newfusepath, - const string &newfusedirpath, - int &error, - vector &tounlink) +_rename_create_path_core(const StrVec &oldbasepaths, + const string &oldbasepath, + const string &newbasepath, + const char *oldfusepath, + const char *newfusepath, + const string &newfusedirpath, + int &error, + StrVec &tounlink) { int rv; bool ismember; @@ -114,19 +112,19 @@ _rename_create_path_core(const vector &oldbasepaths, static int -_rename_create_path(Policy::Func::Search searchFunc, - Policy::Func::Action actionFunc, - const Branches &branches_, +_rename_create_path(const Policy::Search &searchFunc, + const Policy::Action &actionFunc, + const Branches::CPtr &branches_, const char *oldfusepath, const char *newfusepath) { int rv; int error; string newfusedirpath; - vector toremove; - vector newbasepath; - vector oldbasepaths; - vector branches; + StrVec toremove; + StrVec newbasepath; + StrVec oldbasepaths; + StrVec branches; rv = actionFunc(branches_,oldfusepath,&oldbasepaths); if(rv == -1) @@ -134,11 +132,11 @@ _rename_create_path(Policy::Func::Search searchFunc, newfusedirpath = fs::path::dirname(newfusepath); - rv = searchFunc(branches_,newfusedirpath,&newbasepath); + rv = searchFunc(branches_,newfusedirpath.c_str(),&newbasepath); if(rv == -1) return -errno; - branches_.to_paths(branches); + branches_->to_paths(branches); error = -1; for(size_t i = 0, ei = branches.size(); i != ei; i++) @@ -161,15 +159,15 @@ _rename_create_path(Policy::Func::Search searchFunc, static int -_clonepath(Policy::Func::Search searchFunc, - const Branches &branches_, +_clonepath(const Policy::Search &searchFunc, + const Branches::CPtr &branches_, const string &dstbasepath, const string &fusedirpath) { int rv; - vector srcbasepath; + StrVec srcbasepath; - rv = searchFunc(branches_,fusedirpath,&srcbasepath); + rv = searchFunc(branches_,fusedirpath.c_str(),&srcbasepath); if(rv == -1) return -errno; @@ -180,20 +178,20 @@ _clonepath(Policy::Func::Search searchFunc, static int -_clonepath_if_would_create(Policy::Func::Search searchFunc, - Policy::Func::Create createFunc, - const Branches &branches_, +_clonepath_if_would_create(const Policy::Search &searchFunc, + const Policy::Create &createFunc, + const Branches::CPtr &branches_, const string &oldbasepath, const char *oldfusepath, const char *newfusepath) { int rv; string newfusedirpath; - vector newbasepath; + StrVec newbasepath; newfusedirpath = fs::path::dirname(newfusepath); - rv = createFunc(branches_,newfusedirpath,&newbasepath); + rv = createFunc(branches_,newfusedirpath.c_str(),&newbasepath); if(rv == -1) return rv; @@ -205,15 +203,15 @@ _clonepath_if_would_create(Policy::Func::Search searchFunc, static void -_rename_preserve_path_core(Policy::Func::Search searchFunc, - Policy::Func::Create createFunc, - const Branches &branches_, - const vector &oldbasepaths, +_rename_preserve_path_core(const Policy::Search &searchFunc, + const Policy::Create &createFunc, + const Branches::CPtr &branches_, + const StrVec &oldbasepaths, const string &oldbasepath, const char *oldfusepath, const char *newfusepath, int &error, - vector &toremove) + StrVec &toremove) { int rv; bool ismember; @@ -250,24 +248,24 @@ _rename_preserve_path_core(Policy::Func::Search searchFunc, static int -_rename_preserve_path(Policy::Func::Search searchFunc, - Policy::Func::Action actionFunc, - Policy::Func::Create createFunc, - const Branches &branches_, +_rename_preserve_path(const Policy::Search &searchFunc, + const Policy::Action &actionFunc, + const Policy::Create &createFunc, + const Branches::CPtr &branches_, const char *oldfusepath, const char *newfusepath) { int rv; int error; - vector toremove; - vector oldbasepaths; - vector branches; + StrVec toremove; + StrVec oldbasepaths; + StrVec branches; rv = actionFunc(branches_,oldfusepath,&oldbasepaths); if(rv == -1) return -errno; - branches_.to_paths(branches); + branches_->to_paths(branches); error = -1; for(size_t i = 0, ei = branches.size(); i != ei; i++) @@ -293,23 +291,21 @@ namespace FUSE rename(const char *oldpath, const char *newpath) { - const fuse_context *fc = fuse_get_context(); - Config &config = Config::rw(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - config.open_cache.erase(oldpath); - - if(config.func.create.policy->path_preserving() && !config.ignorepponrename) - return _rename_preserve_path(config.func.getattr.policy, - config.func.rename.policy, - config.func.create.policy, - config.branches, + if(cfg->func.create.policy.path_preserving() && !cfg->ignorepponrename) + return _rename_preserve_path(cfg->func.getattr.policy, + cfg->func.rename.policy, + cfg->func.create.policy, + cfg->branches, oldpath, newpath); - return _rename_create_path(config.func.getattr.policy, - config.func.rename.policy, - config.branches, + return _rename_create_path(cfg->func.getattr.policy, + cfg->func.rename.policy, + cfg->branches, oldpath, newpath); } diff --git a/src/fuse_rename.hpp b/src/fuse_rename.hpp index affa01d2..abbcf515 100644 --- a/src/fuse_rename.hpp +++ b/src/fuse_rename.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_rmdir.cpp b/src/fuse_rmdir.cpp index 9c650caf..c7c6e107 100644 --- a/src/fuse_rmdir.cpp +++ b/src/fuse_rmdir.cpp @@ -20,7 +20,7 @@ #include "fs_path.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include @@ -66,8 +66,8 @@ namespace l static int - rmdir_loop(const vector &basepaths_, - const char *fusepath_) + rmdir_loop(const StrVec &basepaths_, + const char *fusepath_) { int error; @@ -82,7 +82,7 @@ namespace l static int - rmdir(Policy::Func::Action actionFunc_, + rmdir(const Policy::Action &actionFunc_, const Branches &branches_, const char *fusepath_) { @@ -102,12 +102,12 @@ namespace FUSE int rmdir(const char *fusepath_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::rmdir(config.func.rmdir.policy, - config.branches, + return l::rmdir(cfg->func.rmdir.policy, + cfg->branches, fusepath_); } } diff --git a/src/fuse_rmdir.hpp b/src/fuse_rmdir.hpp index 0c59659c..d2782468 100644 --- a/src/fuse_rmdir.hpp +++ b/src/fuse_rmdir.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_setxattr.cpp b/src/fuse_setxattr.cpp index 6bb9d041..c46bc7c1 100644 --- a/src/fuse_setxattr.cpp +++ b/src/fuse_setxattr.cpp @@ -25,7 +25,7 @@ #include "str.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -69,31 +69,30 @@ namespace l static int - setxattr_controlfile(Config &config_, - const string &attrname_, + setxattr_controlfile(const string &attrname_, const string &attrval_, const int flags_) { int rv; string key; + Config::Write cfg; if(!str::startswith(attrname_,"user.mergerfs.")) return -ENOATTR; key = &attrname_[14]; - if(config_.has_key(key) == false) + if(cfg->has_key(key) == false) return -ENOATTR; if((flags_ & XATTR_CREATE) == XATTR_CREATE) return -EEXIST; - rv = config_.set(key,attrval_); + rv = cfg->set(key,attrval_); if(rv < 0) return rv; - config_.open_cache.clear(); - fs::statvfs_cache_timeout(config_.cache_statfs); + fs::statvfs_cache_timeout(cfg->cache_statfs); return rv; } @@ -120,13 +119,13 @@ namespace l static void - setxattr_loop(const vector &basepaths_, - const char *fusepath_, - const char *attrname_, - const char *attrval_, - const size_t attrvalsize_, - const int flags_, - PolicyRV *prv_) + setxattr_loop(const StrVec &basepaths_, + const char *fusepath_, + const char *attrname_, + const char *attrval_, + const size_t attrvalsize_, + const int flags_, + PolicyRV *prv_) { for(size_t i = 0, ei = basepaths_.size(); i != ei; i++) { @@ -138,8 +137,8 @@ namespace l static int - setxattr(Policy::Func::Action actionFunc_, - Policy::Func::Search searchFunc_, + setxattr(const Policy::Action &setxattrPolicy_, + const Policy::Search &getxattrPolicy_, const Branches &branches_, const char *fusepath_, const char *attrname_, @@ -149,9 +148,9 @@ namespace l { int rv; PolicyRV prv; - vector basepaths; + StrVec basepaths; - rv = actionFunc_(branches_,fusepath_,&basepaths); + rv = setxattrPolicy_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; @@ -162,16 +161,13 @@ namespace l return prv.error[0].rv; basepaths.clear(); - rv = searchFunc_(branches_,fusepath_,&basepaths); + rv = getxattrPolicy_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; return l::get_error(prv,basepaths[0]); } -} -namespace FUSE -{ int setxattr(const char *fusepath_, const char *attrname_, @@ -179,27 +175,21 @@ namespace FUSE size_t attrvalsize_, int flags_) { - Config &config = Config::rw(); - - if(fusepath_ == config.controlfile) - return l::setxattr_controlfile(config, - attrname_, - string(attrval_,attrvalsize_), - flags_); + Config::Read cfg; - if((config.security_capability == false) && + if((cfg->security_capability == false) && l::is_attrname_security_capability(attrname_)) return -ENOATTR; - if(config.xattr.to_int()) - return -config.xattr.to_int(); + if(cfg->xattr.to_int()) + return -cfg->xattr.to_int(); const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::setxattr(config.func.setxattr.policy, - config.func.getxattr.policy, - config.branches, + return l::setxattr(cfg->func.setxattr.policy, + cfg->func.getxattr.policy, + cfg->branches, fusepath_, attrname_, attrval_, @@ -207,3 +197,21 @@ namespace FUSE flags_); } } + +namespace FUSE +{ + int + setxattr(const char *fusepath_, + const char *attrname_, + const char *attrval_, + size_t attrvalsize_, + int flags_) + { + if(fusepath_ == CONTROLFILE) + return l::setxattr_controlfile(attrname_, + string(attrval_,attrvalsize_), + flags_); + + return l::setxattr(fusepath_,attrname_,attrval_,attrvalsize_,flags_); + } +} diff --git a/src/fuse_setxattr.hpp b/src/fuse_setxattr.hpp index 2b882b66..6ae1f52e 100644 --- a/src/fuse_setxattr.hpp +++ b/src/fuse_setxattr.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_statfs.cpp b/src/fuse_statfs.cpp index 0c15f7bf..68ffb2cf 100644 --- a/src/fuse_statfs.cpp +++ b/src/fuse_statfs.cpp @@ -19,11 +19,10 @@ #include "fs_lstat.hpp" #include "fs_path.hpp" #include "fs_statvfs.hpp" -#include "rwlock.hpp" #include "statvfs_util.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -35,6 +34,7 @@ using std::string; using std::map; using std::vector; + namespace l { static @@ -69,24 +69,22 @@ namespace l static bool should_ignore(const StatFSIgnore ignore_, - const Branch *branch_, + const Branch &branch_, const bool readonly_) { return ((((ignore_ == StatFSIgnore::ENUM::RO) || readonly_) && - (branch_->ro_or_nc())) || - ((ignore_ == StatFSIgnore::ENUM::NC) && (branch_->nc()))); + (branch_.ro_or_nc())) || + ((ignore_ == StatFSIgnore::ENUM::NC) && (branch_.nc()))); } static int - statfs(const Branches &branches_, - const char *fusepath_, - const StatFS mode_, - const StatFSIgnore ignore_, - struct statvfs *fsstat_) + statfs(const Branches::CPtr &branches_, + const char *fusepath_, + const StatFS mode_, + const StatFSIgnore ignore_, + struct statvfs *fsstat_) { - rwlock::ReadGuard guard(branches_.lock); - int rv; string fullpath; struct stat st; @@ -99,11 +97,11 @@ namespace l min_bsize = std::numeric_limits::max(); min_frsize = std::numeric_limits::max(); min_namemax = std::numeric_limits::max(); - for(size_t i = 0, ei = branches_.vec.size(); i < ei; i++) + for(const auto &branch : *branches_) { fullpath = ((mode_ == StatFS::ENUM::FULL) ? - fs::path::make(branches_.vec[i].path,fusepath_) : - branches_.vec[i].path); + fs::path::make(branch.path,fusepath_) : + branch.path); rv = fs::lstat(fullpath,&st); if(rv == -1) @@ -120,7 +118,7 @@ namespace l if(stvfs.f_namemax && (min_namemax > stvfs.f_namemax)) min_namemax = stvfs.f_namemax; - if(l::should_ignore(ignore_,&branches_.vec[i],StatVFS::readonly(stvfs))) + if(l::should_ignore(ignore_,branch,StatVFS::readonly(stvfs))) { stvfs.f_bavail = 0; stvfs.f_favail = 0; @@ -153,14 +151,14 @@ namespace FUSE statfs(const char *fusepath_, struct statvfs *st_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::statfs(config.branches, + return l::statfs(cfg->branches, fusepath_, - config.statfs, - config.statfs_ignore, + cfg->statfs, + cfg->statfs_ignore, st_); } } diff --git a/src/fuse_statfs.hpp b/src/fuse_statfs.hpp index 8d0933bb..c34ac43a 100644 --- a/src/fuse_statfs.hpp +++ b/src/fuse_statfs.hpp @@ -18,6 +18,7 @@ #include + namespace FUSE { int diff --git a/src/fuse_symlink.cpp b/src/fuse_symlink.cpp index c332d6f4..cdfb407c 100644 --- a/src/fuse_symlink.cpp +++ b/src/fuse_symlink.cpp @@ -21,7 +21,7 @@ #include "fs_path.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include @@ -29,7 +29,6 @@ #include using std::string; -using std::vector; namespace error @@ -73,11 +72,11 @@ namespace l static int - symlink_loop(const string &existingpath_, - const vector &newbasepaths_, - const char *oldpath_, - const char *newpath_, - const string &newdirpath_) + symlink_loop(const string &existingpath_, + const StrVec &newbasepaths_, + const char *oldpath_, + const char *newpath_, + const string &newdirpath_) { int rv; int error; @@ -100,16 +99,16 @@ namespace l static int - symlink(Policy::Func::Search searchFunc_, - Policy::Func::Create createFunc_, + symlink(const Policy::Search &searchFunc_, + const Policy::Create &createFunc_, const Branches &branches_, const char *oldpath_, const char *newpath_) { int rv; string newdirpath; - vector newbasepaths; - vector existingpaths; + StrVec newbasepaths; + StrVec existingpaths; newdirpath = fs::path::dirname(newpath_); @@ -132,13 +131,13 @@ namespace FUSE symlink(const char *oldpath_, const char *newpath_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::symlink(config.func.getattr.policy, - config.func.symlink.policy, - config.branches, + return l::symlink(cfg->func.getattr.policy, + cfg->func.symlink.policy, + cfg->branches, oldpath_, newpath_); } diff --git a/src/fuse_symlink.hpp b/src/fuse_symlink.hpp index 561a8300..0bbced2e 100644 --- a/src/fuse_symlink.hpp +++ b/src/fuse_symlink.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_truncate.cpp b/src/fuse_truncate.cpp index 331f0c4d..669715a8 100644 --- a/src/fuse_truncate.cpp +++ b/src/fuse_truncate.cpp @@ -21,16 +21,14 @@ #include "policy_rv.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include -#include #include #include using std::string; -using std::vector; namespace l @@ -74,10 +72,10 @@ namespace l static void - truncate_loop(const vector &basepaths_, - const char *fusepath_, - const off_t size_, - PolicyRV *prv_) + truncate_loop(const StrVec &basepaths_, + const char *fusepath_, + const off_t size_, + PolicyRV *prv_) { for(size_t i = 0, ei = basepaths_.size(); i != ei; i++) { @@ -87,15 +85,15 @@ namespace l static int - truncate(Policy::Func::Action actionFunc_, - Policy::Func::Search searchFunc_, + truncate(const Policy::Action &actionFunc_, + const Policy::Search &searchFunc_, const Branches &branches_, const char *fusepath_, const off_t size_) { int rv; PolicyRV prv; - vector basepaths; + StrVec basepaths; rv = actionFunc_(branches_,fusepath_,&basepaths); if(rv == -1) @@ -122,13 +120,13 @@ namespace FUSE truncate(const char *fusepath_, off_t size_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::truncate(config.func.truncate.policy, - config.func.getattr.policy, - config.branches, + return l::truncate(cfg->func.truncate.policy, + cfg->func.getattr.policy, + cfg->branches, fusepath_, size_); } diff --git a/src/fuse_truncate.hpp b/src/fuse_truncate.hpp index ea7557db..61331295 100644 --- a/src/fuse_truncate.hpp +++ b/src/fuse_truncate.hpp @@ -18,6 +18,7 @@ #include + namespace FUSE { int diff --git a/src/fuse_unlink.cpp b/src/fuse_unlink.cpp index 507590c1..5f551935 100644 --- a/src/fuse_unlink.cpp +++ b/src/fuse_unlink.cpp @@ -20,7 +20,7 @@ #include "fs_unlink.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include #include @@ -84,14 +84,14 @@ namespace l static int - unlink(Policy::Func::Action actionFunc_, + unlink(const Policy::Action &unlinkPolicy_, const Branches &branches_, const char *fusepath_) { int rv; vector basepaths; - rv = actionFunc_(branches_,fusepath_,&basepaths); + rv = unlinkPolicy_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; @@ -104,14 +104,12 @@ namespace FUSE int unlink(const char *fusepath_) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - config.open_cache.erase(fusepath_); - - return l::unlink(config.func.unlink.policy, - config.branches, + return l::unlink(cfg->func.unlink.policy, + cfg->branches, fusepath_); } } diff --git a/src/fuse_unlink.hpp b/src/fuse_unlink.hpp index e7f1a272..845ac186 100644 --- a/src/fuse_unlink.hpp +++ b/src/fuse_unlink.hpp @@ -16,6 +16,7 @@ #pragma once + namespace FUSE { int diff --git a/src/fuse_utimens.cpp b/src/fuse_utimens.cpp index e5a072ac..83c4b584 100644 --- a/src/fuse_utimens.cpp +++ b/src/fuse_utimens.cpp @@ -21,15 +21,13 @@ #include "policy_rv.hpp" #include "ugid.hpp" -#include +#include "fuse.h" #include -#include #include using std::string; -using std::vector; namespace l @@ -73,10 +71,10 @@ namespace l static void - utimens_loop(const vector &basepaths_, - const char *fusepath_, - const timespec ts_[2], - PolicyRV *prv_) + utimens_loop(const StrVec &basepaths_, + const char *fusepath_, + const timespec ts_[2], + PolicyRV *prv_) { for(size_t i = 0, ei = basepaths_.size(); i != ei; i++) { @@ -86,17 +84,17 @@ namespace l static int - utimens(Policy::Func::Action actionFunc_, - Policy::Func::Search searchFunc_, + utimens(const Policy::Action &utimensPolicy_, + const Policy::Search &getattrPolicy_, const Branches &branches_, const char *fusepath_, const timespec ts_[2]) { int rv; PolicyRV prv; - vector basepaths; + StrVec basepaths; - rv = actionFunc_(branches_,fusepath_,&basepaths); + rv = utimensPolicy_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; @@ -107,7 +105,7 @@ namespace l return prv.error[0].rv; basepaths.clear(); - rv = searchFunc_(branches_,fusepath_,&basepaths); + rv = getattrPolicy_(branches_,fusepath_,&basepaths); if(rv == -1) return -errno; @@ -121,13 +119,13 @@ namespace FUSE utimens(const char *fusepath_, const timespec ts_[2]) { - const fuse_context *fc = fuse_get_context(); - const Config &config = Config::ro(); + Config::Read cfg; + const fuse_context *fc = fuse_get_context(); const ugid::Set ugid(fc->uid,fc->gid); - return l::utimens(config.func.utimens.policy, - config.func.getattr.policy, - config.branches, + return l::utimens(cfg->func.utimens.policy, + cfg->func.getattr.policy, + cfg->branches, fusepath_, ts_); } diff --git a/src/fuse_utimens.hpp b/src/fuse_utimens.hpp index cb78c181..2b24258e 100644 --- a/src/fuse_utimens.hpp +++ b/src/fuse_utimens.hpp @@ -18,6 +18,7 @@ #include + namespace FUSE { int diff --git a/src/fuse_write.cpp b/src/fuse_write.cpp index 9dbad931..303b6cba 100644 --- a/src/fuse_write.cpp +++ b/src/fuse_write.cpp @@ -21,16 +21,17 @@ #include "fs_write.hpp" #include "ugid.hpp" +#include "fuse.h" + #include #include -#include - using std::string; using std::vector; typedef int (*WriteFunc)(const int,const void*,const size_t,const off_t); + namespace l { static @@ -85,13 +86,13 @@ namespace l int err_) { int rv; - const Config &config = Config::ro(); + Config::Read cfg; - if(config.moveonenospc.enabled == false) + if(cfg->moveonenospc.enabled == false) return err_; - rv = fs::movefile_as_root(config.moveonenospc.policy, - config.branches, + rv = fs::movefile_as_root(cfg->moveonenospc.policy, + cfg->branches, fi_->fusepath, &fi_->fd); if(rv == -1) diff --git a/src/fuse_write.hpp b/src/fuse_write.hpp index 7d439b7c..abbfd093 100644 --- a/src/fuse_write.hpp +++ b/src/fuse_write.hpp @@ -16,7 +16,8 @@ #pragma once -#include +#include "fuse.h" + namespace FUSE { diff --git a/src/fuse_write_buf.cpp b/src/fuse_write_buf.cpp index 899e377e..fb06e4de 100644 --- a/src/fuse_write_buf.cpp +++ b/src/fuse_write_buf.cpp @@ -31,6 +31,7 @@ using std::string; using std::vector; + namespace l { static @@ -67,13 +68,13 @@ namespace l int err_) { int rv; - const Config &config = Config::ro(); + Config::Read cfg; - if(config.moveonenospc.enabled == false) + if(cfg->moveonenospc.enabled == false) return err_; - rv = fs::movefile_as_root(config.moveonenospc.policy, - config.branches, + rv = fs::movefile_as_root(cfg->moveonenospc.policy, + cfg->branches, fi_->fusepath, &fi_->fd); if(rv == -1) diff --git a/src/fuse_write_buf.hpp b/src/fuse_write_buf.hpp index cf225918..be4779df 100644 --- a/src/fuse_write_buf.hpp +++ b/src/fuse_write_buf.hpp @@ -16,10 +16,11 @@ #pragma once -#include +#include "fuse.h" #include + namespace FUSE { int diff --git a/src/hw_cpu.cpp b/src/hw_cpu.cpp index c65e713b..b829c2fc 100644 --- a/src/hw_cpu.cpp +++ b/src/hw_cpu.cpp @@ -18,6 +18,7 @@ #include + namespace hw { namespace cpu diff --git a/src/hw_cpu.hpp b/src/hw_cpu.hpp index dd17c1de..7c702219 100644 --- a/src/hw_cpu.hpp +++ b/src/hw_cpu.hpp @@ -18,6 +18,7 @@ #pragma once + namespace hw { namespace cpu diff --git a/src/mergerfs.cpp b/src/mergerfs.cpp index a5b65a15..11c7fc67 100644 --- a/src/mergerfs.cpp +++ b/src/mergerfs.cpp @@ -18,6 +18,7 @@ #include "mergerfs.hpp" #include "option_parser.hpp" #include "resources.hpp" +#include "strvec.hpp" #include "fuse_access.hpp" #include "fuse_chmod.hpp" @@ -66,13 +67,14 @@ #include "fuse_write.hpp" #include "fuse_write_buf.hpp" -#include +#include "fuse.h" #include #include #include + namespace l { static @@ -151,10 +153,10 @@ namespace l main(const int argc_, char **argv_) { - Config config; - fuse_args args; - fuse_operations ops; - std::vector errs; + Config::Read cfg; + Config::ErrVec errs; + fuse_args args; + fuse_operations ops; memset(&ops,0,sizeof(fuse_operations)); @@ -162,22 +164,17 @@ namespace l args.argv = argv_; args.allocated = 0; - options::parse(&args,&config,&errs); + options::parse(&args,&errs); if(errs.size()) - { - for(uint64_t i = 0; i < errs.size(); i++) - std::cerr << "* ERROR: " << errs[i] << std::endl; - - return 1; - } + std::cerr << errs << std::endl; l::setup_resources(); - l::get_fuse_operations(ops,config.nullrw); + l::get_fuse_operations(ops,cfg->nullrw); return fuse_main(args.argc, args.argv, &ops, - &config); + NULL); } } diff --git a/src/num.cpp b/src/num.cpp index 67ad43b7..667be092 100644 --- a/src/num.cpp +++ b/src/num.cpp @@ -16,9 +16,9 @@ #include "ef.hpp" +#include #include -#include #include #include #include @@ -28,6 +28,7 @@ #define GB (MB * 1024UL) #define TB (GB * 1024UL) + namespace num { int diff --git a/src/num.hpp b/src/num.hpp index 40ba30c2..4e8dc79e 100644 --- a/src/num.hpp +++ b/src/num.hpp @@ -16,10 +16,11 @@ #pragma once -#include - +#include #include +#include + namespace num { int to_uint64_t(const std::string &str, uint64_t *value); diff --git a/src/option_parser.cpp b/src/option_parser.cpp index 5646ee14..519b52c3 100644 --- a/src/option_parser.cpp +++ b/src/option_parser.cpp @@ -25,7 +25,7 @@ #include "str.hpp" #include "version.hpp" -#include +#include "fuse.h" #include #include @@ -48,12 +48,6 @@ enum MERGERFS_OPT_VERSION }; -struct Data -{ - Config *config; - vector *errs; -}; - namespace l { @@ -74,140 +68,81 @@ namespace l return tc; } - - static - void - read_config(Data *data_, - std::istream &is_) - { - int rv; - std::string key; - std::string val; - std::string line; - - rv = 0; - while(std::getline(is_,line,'\n')) - { - line = str::trim(line); - if(!line.empty() && (line[0] == '#')) - continue; - - str::splitkv(line,'=',&key,&val); - key = str::trim(key); - val = str::trim(val); - - if(key.empty() || val.empty()) - continue; - - rv = data_->config->set_raw(key,val); - switch(rv) - { - case -EINVAL: - data_->errs->push_back("invalid argument - " + line); - break; - case -ENOATTR: - data_->errs->push_back("unknown option - " + line); - break; - default: - break; - } - } - } - - static - void - read_config(Data *data_, - const std::string &filepath_) - { - std::ifstream is; - - is.open(filepath_); - if(is.bad()) - { - data_->errs->push_back("unable to open config - " + filepath_); - } - else - { - l::read_config(data_,is); - if(!is.eof()) - data_->errs->push_back("failure reading config file"); - is.close(); - } - } } static void -set_option(fuse_args *args, - const std::string &option_) +set_option(const std::string &option_, + fuse_args *args_) { - fuse_opt_add_arg(args,"-o"); - fuse_opt_add_arg(args,option_.c_str()); + fuse_opt_add_arg(args_,"-o"); + fuse_opt_add_arg(args_,option_.c_str()); } static void -set_kv_option(fuse_args *args, - const std::string &key, - const std::string &value) +set_kv_option(const std::string &key_, + const std::string &val_, + fuse_args *args_) { std::string option; - option = key + '=' + value; + option = key_ + '=' + val_; - set_option(args,option); + set_option(option,args_); } static void -set_threads(fuse_args *args_, - Config *config_) +set_threads(Config::Write &cfg_, + fuse_args *args_) { int threads; - threads = l::calculate_thread_count(config_->threads); + threads = l::calculate_thread_count(cfg_->threads); - config_->threads = threads; + cfg_->threads = threads; - set_kv_option(args_,"threads",config_->threads.to_string()); + set_kv_option("threads",cfg_->threads.to_string(),args_); } static void -set_fsname(fuse_args *args_, - Config *config_) +set_fsname(Config::Write &cfg_, + fuse_args *args_) { - if(config_->fsname->empty()) + if(cfg_->fsname->empty()) { - vector branches; + vector paths; - config_->branches.to_paths(branches); + cfg_->branches->to_paths(paths); - if(branches.size() > 0) - config_->fsname = str::remove_common_prefix_and_join(branches,':'); + if(paths.size() > 0) + cfg_->fsname = str::remove_common_prefix_and_join(paths,':'); } - set_kv_option(args_,"fsname",config_->fsname); + set_kv_option("fsname",cfg_->fsname,args_); } static void -set_subtype(fuse_args *args) +set_subtype(fuse_args *args_) { - set_kv_option(args,"subtype","mergerfs"); + set_kv_option("subtype","mergerfs",args_); } static void -set_default_options(fuse_args *args) +set_default_options(fuse_args *args_) { - set_option(args,"default_permissions"); + set_option("default_permissions",args_); } static int -parse_and_process_kv_arg(Data *data_, +parse_and_process_kv_arg(Config::Write &cfg_, + Config::ErrVec *errs_, const std::string &key_, const std::string &val_) { @@ -217,7 +152,7 @@ parse_and_process_kv_arg(Data *data_, rv = 0; if(key == "config") - return (l::read_config(data_,val_),0); + return ((cfg_->from_file(val_,errs_) < 0) ? 1 : 0); ef(key == "attr_timeout") key = "cache.attr"; ef(key == "entry_timeout") @@ -245,57 +180,62 @@ parse_and_process_kv_arg(Data *data_, ef(key == "cache.open") return 0; - if(data_->config->has_key(key) == false) + if(cfg_->has_key(key) == false) return 1; - rv = data_->config->set_raw(key,val); + rv = cfg_->set_raw(key,val); if(rv) - data_->errs->push_back("invalid argument - " + key_ + '=' + val_); + errs_->push_back({rv,key+'='+val}); return 0; } static int -process_opt(Data *data_, +process_opt(Config::Write &cfg_, + Config::ErrVec *errs_, const std::string &arg_) { std::string key; std::string val; str::splitkv(arg_,'=',&key,&val); + key = str::trim(key); + val = str::trim(val); - return parse_and_process_kv_arg(data_,key,val); + return parse_and_process_kv_arg(cfg_,errs_,key,val); } static int -process_branches(Data *data_, - const char *arg_) +process_branches(Config::Write &cfg_, + Config::ErrVec *errs_, + const char *arg_) { int rv; string arg; arg = arg_; - rv = data_->config->set_raw("branches",arg); + rv = cfg_->set_raw("branches",arg); if(rv) - data_->errs->push_back("unable to parse 'branches' - " + arg); + errs_->push_back({rv,"branches="+arg}); return 0; } static int -process_mount(Data *data_, - const char *arg_) +process_mount(Config::Write &cfg_, + Config::ErrVec *errs_, + const char *arg_) { int rv; string arg; arg = arg_; - rv = data_->config->set_raw("mount",arg); + rv = cfg_->set_raw("mount",arg); if(rv) - data_->errs->push_back("unable to set 'mount' - " + arg); + errs_->push_back({rv,"mount="+arg}); return 1; } @@ -412,18 +352,19 @@ option_processor(void *data_, int key_, fuse_args *outargs_) { - Data *data = (Data*)data_; + Config::Write cfg; + Config::ErrVec *errs = (Config::ErrVec*)data_; switch(key_) { case FUSE_OPT_KEY_OPT: - return process_opt(data,arg_); + return process_opt(cfg,errs,arg_); case FUSE_OPT_KEY_NONOPT: - if(data->config->branches.vec.empty()) - return process_branches(data,arg_); + if(cfg->branches->empty()) + return process_branches(cfg,errs,arg_); else - return process_mount(data,arg_); + return process_mount(cfg,errs,arg_); case MERGERFS_OPT_HELP: usage(); @@ -445,11 +386,10 @@ option_processor(void *data_, namespace options { void - parse(fuse_args *args_, - Config *config_, - std::vector *errs_) + parse(fuse_args *args_, + Config::ErrVec *errs_) { - Data data; + Config::Write cfg; const struct fuse_opt opts[] = { FUSE_OPT_KEY("-h",MERGERFS_OPT_HELP), @@ -460,21 +400,19 @@ namespace options {NULL,-1U,0} }; - data.config = config_; - data.errs = errs_; fuse_opt_parse(args_, - &data, + errs_, opts, ::option_processor); - if(config_->branches.vec.empty()) - errs_->push_back("branches not set"); - if(config_->mount->empty()) - errs_->push_back("mountpoint not set"); + if(cfg->branches->empty()) + errs_->push_back({0,"branches not set"}); + if(cfg->mount->empty()) + errs_->push_back({0,"mountpoint not set"}); set_default_options(args_); - set_fsname(args_,config_); + set_fsname(cfg,args_); set_subtype(args_); - set_threads(args_,config_); + set_threads(cfg,args_); } } diff --git a/src/option_parser.hpp b/src/option_parser.hpp index 324eb905..1e9c4122 100644 --- a/src/option_parser.hpp +++ b/src/option_parser.hpp @@ -18,15 +18,12 @@ #include "config.hpp" -#include -#include +#include "fuse.h" -#include namespace options { void - parse(fuse_args *args, - Config *config, - std::vector *errs); + parse(fuse_args *args, + Config::ErrVec *errs); } diff --git a/src/policies.cpp b/src/policies.cpp new file mode 100644 index 00000000..dc64370c --- /dev/null +++ b/src/policies.cpp @@ -0,0 +1,132 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#include "policies.hpp" + +#define IFERTA(X) if(name_ == #X) return &Policies::Action::X; +#define IFERTC(X) if(name_ == #X) return &Policies::Create::X; +#define IFERTS(X) if(name_ == #X) return &Policies::Search::X; + +#define IFERT(FUNC) \ + FUNC(all) \ + FUNC(epall) \ + FUNC(epff) \ + FUNC(eplfs) \ + FUNC(eplus) \ + FUNC(epmfs) \ + FUNC(eppfrd) \ + FUNC(eprand) \ + FUNC(erofs) \ + FUNC(ff) \ + FUNC(lfs) \ + FUNC(lus) \ + FUNC(mfs) \ + FUNC(msplfs) \ + FUNC(msplus) \ + FUNC(mspmfs) \ + FUNC(msppfrd) \ + FUNC(newest) \ + FUNC(pfrd) \ + FUNC(rand) + +Policy::ActionImpl* +Policies::Action::find(const std::string &name_) +{ + IFERT(IFERTA); + + return NULL; +} + +Policy::CreateImpl* +Policies::Create::find(const std::string &name_) +{ + IFERT(IFERTC); + + return NULL; +} + +Policy::SearchImpl* +Policies::Search::find(const std::string &name_) +{ + IFERT(IFERTS); + + return NULL; +} + +Policy::All::Action Policies::Action::all; +Policy::EPAll::Action Policies::Action::epall; +Policy::EPFF::Action Policies::Action::epff; +Policy::EPLFS::Action Policies::Action::eplfs; +Policy::EPLUS::Action Policies::Action::eplus; +Policy::EPMFS::Action Policies::Action::epmfs; +Policy::EPPFRD::Action Policies::Action::eppfrd; +Policy::EPRand::Action Policies::Action::eprand; +Policy::ERoFS::Action Policies::Action::erofs; +Policy::FF::Action Policies::Action::ff; +Policy::LFS::Action Policies::Action::lfs; +Policy::LUS::Action Policies::Action::lus; +Policy::MFS::Action Policies::Action::mfs; +Policy::MSPLFS::Action Policies::Action::msplfs; +Policy::MSPLUS::Action Policies::Action::msplus; +Policy::MSPMFS::Action Policies::Action::mspmfs; +Policy::MSPPFRD::Action Policies::Action::msppfrd; +Policy::Newest::Action Policies::Action::newest; +Policy::PFRD::Action Policies::Action::pfrd; +Policy::Rand::Action Policies::Action::rand; + +Policy::All::Create Policies::Create::all; +Policy::EPAll::Create Policies::Create::epall; +Policy::EPFF::Create Policies::Create::epff; +Policy::EPLFS::Create Policies::Create::eplfs; +Policy::EPLUS::Create Policies::Create::eplus; +Policy::EPMFS::Create Policies::Create::epmfs; +Policy::EPPFRD::Create Policies::Create::eppfrd; +Policy::EPRand::Create Policies::Create::eprand; +Policy::ERoFS::Create Policies::Create::erofs; +Policy::FF::Create Policies::Create::ff; +Policy::LFS::Create Policies::Create::lfs; +Policy::LUS::Create Policies::Create::lus; +Policy::MFS::Create Policies::Create::mfs; +Policy::MSPLFS::Create Policies::Create::msplfs; +Policy::MSPLUS::Create Policies::Create::msplus; +Policy::MSPMFS::Create Policies::Create::mspmfs; +Policy::MSPPFRD::Create Policies::Create::msppfrd; +Policy::Newest::Create Policies::Create::newest; +Policy::PFRD::Create Policies::Create::pfrd; +Policy::Rand::Create Policies::Create::rand; + +Policy::All::Search Policies::Search::all; +Policy::EPAll::Search Policies::Search::epall; +Policy::EPFF::Search Policies::Search::epff; +Policy::EPLFS::Search Policies::Search::eplfs; +Policy::EPLUS::Search Policies::Search::eplus; +Policy::EPMFS::Search Policies::Search::epmfs; +Policy::EPPFRD::Search Policies::Search::eppfrd; +Policy::EPRand::Search Policies::Search::eprand; +Policy::ERoFS::Search Policies::Search::erofs; +Policy::FF::Search Policies::Search::ff; +Policy::LFS::Search Policies::Search::lfs; +Policy::LUS::Search Policies::Search::lus; +Policy::MFS::Search Policies::Search::mfs; +Policy::MSPLFS::Search Policies::Search::msplfs; +Policy::MSPLUS::Search Policies::Search::msplus; +Policy::MSPMFS::Search Policies::Search::mspmfs; +Policy::MSPPFRD::Search Policies::Search::msppfrd; +Policy::Newest::Search Policies::Search::newest; +Policy::PFRD::Search Policies::Search::pfrd; +Policy::Rand::Search Policies::Search::rand; diff --git a/src/policies.hpp b/src/policies.hpp new file mode 100644 index 00000000..65d7f7a0 --- /dev/null +++ b/src/policies.hpp @@ -0,0 +1,121 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy_all.hpp" +#include "policy_epall.hpp" +#include "policy_epff.hpp" +#include "policy_eplfs.hpp" +#include "policy_eplus.hpp" +#include "policy_epmfs.hpp" +#include "policy_eppfrd.hpp" +#include "policy_eprand.hpp" +#include "policy_erofs.hpp" +#include "policy_ff.hpp" +#include "policy_lfs.hpp" +#include "policy_lus.hpp" +#include "policy_mfs.hpp" +#include "policy_msplfs.hpp" +#include "policy_msplus.hpp" +#include "policy_mspmfs.hpp" +#include "policy_msppfrd.hpp" +#include "policy_newest.hpp" +#include "policy_pfrd.hpp" +#include "policy_rand.hpp" + +struct Policies +{ + struct Action + { + static Policy::ActionImpl *find(const std::string &name); + + static Policy::All::Action all; + static Policy::EPAll::Action epall; + static Policy::EPFF::Action epff; + static Policy::EPLFS::Action eplfs; + static Policy::EPLUS::Action eplus; + static Policy::EPMFS::Action epmfs; + static Policy::EPPFRD::Action eppfrd; + static Policy::EPRand::Action eprand; + static Policy::ERoFS::Action erofs; + static Policy::FF::Action ff; + static Policy::LFS::Action lfs; + static Policy::LUS::Action lus; + static Policy::MFS::Action mfs; + static Policy::MSPLFS::Action msplfs; + static Policy::MSPLUS::Action msplus; + static Policy::MSPMFS::Action mspmfs; + static Policy::MSPPFRD::Action msppfrd; + static Policy::Newest::Action newest; + static Policy::PFRD::Action pfrd; + static Policy::Rand::Action rand; + }; + + struct Create + { + static Policy::CreateImpl *find(const std::string &name); + + static Policy::All::Create all; + static Policy::EPAll::Create epall; + static Policy::EPFF::Create epff; + static Policy::EPLFS::Create eplfs; + static Policy::EPLUS::Create eplus; + static Policy::EPMFS::Create epmfs; + static Policy::EPPFRD::Create eppfrd; + static Policy::EPRand::Create eprand; + static Policy::ERoFS::Create erofs; + static Policy::FF::Create ff; + static Policy::LFS::Create lfs; + static Policy::LUS::Create lus; + static Policy::MFS::Create mfs; + static Policy::MSPLFS::Create msplfs; + static Policy::MSPLUS::Create msplus; + static Policy::MSPMFS::Create mspmfs; + static Policy::MSPPFRD::Create msppfrd; + static Policy::Newest::Create newest; + static Policy::PFRD::Create pfrd; + static Policy::Rand::Create rand; + }; + + struct Search + { + static Policy::SearchImpl *find(const std::string &name); + + static Policy::All::Search all; + static Policy::EPAll::Search epall; + static Policy::EPFF::Search epff; + static Policy::EPLFS::Search eplfs; + static Policy::EPLUS::Search eplus; + static Policy::EPMFS::Search epmfs; + static Policy::EPPFRD::Search eppfrd; + static Policy::EPRand::Search eprand; + static Policy::ERoFS::Search erofs; + static Policy::FF::Search ff; + static Policy::LFS::Search lfs; + static Policy::LUS::Search lus; + static Policy::MFS::Search mfs; + static Policy::MSPLFS::Search msplfs; + static Policy::MSPLUS::Search msplus; + static Policy::MSPMFS::Search mspmfs; + static Policy::MSPPFRD::Search msppfrd; + static Policy::Newest::Search newest; + static Policy::PFRD::Search pfrd; + static Policy::Rand::Search rand; + }; +}; diff --git a/src/policy.cpp b/src/policy.cpp deleted file mode 100644 index 3e854bad..00000000 --- a/src/policy.cpp +++ /dev/null @@ -1,96 +0,0 @@ -/* - Copyright (c) 2016, Antonio SJ Musumeci - - Permission to use, copy, modify, and/or distribute this software for any - purpose with or without fee is hereby granted, provided that the above - copyright notice and this permission notice appear in all copies. - - THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES - WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF - MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR - ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES - WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN - ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF - OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. -*/ - -#include - -#include "buildvector.hpp" -#include "policy.hpp" - -#define POLICY(X,PP) (Policy(Policy::Enum::X,#X,Policy::Func::X,PP)) -#define PRESERVES_PATH true -#define DOESNT_PRESERVE_PATH false - -const std::vector Policy::_policies_ = - buildvector - (POLICY(invalid,DOESNT_PRESERVE_PATH)) - (POLICY(all,DOESNT_PRESERVE_PATH)) - (POLICY(epall,PRESERVES_PATH)) - (POLICY(epff,PRESERVES_PATH)) - (POLICY(eplfs,PRESERVES_PATH)) - (POLICY(eplus,PRESERVES_PATH)) - (POLICY(epmfs,PRESERVES_PATH)) - (POLICY(eppfrd,PRESERVES_PATH)) - (POLICY(eprand,PRESERVES_PATH)) - (POLICY(erofs,DOESNT_PRESERVE_PATH)) - (POLICY(ff,DOESNT_PRESERVE_PATH)) - (POLICY(lfs,DOESNT_PRESERVE_PATH)) - (POLICY(lus,DOESNT_PRESERVE_PATH)) - (POLICY(mfs,DOESNT_PRESERVE_PATH)) - (POLICY(msplfs,PRESERVES_PATH)) - (POLICY(msplus,PRESERVES_PATH)) - (POLICY(mspmfs,PRESERVES_PATH)) - (POLICY(msppfrd,PRESERVES_PATH)) - (POLICY(newest,DOESNT_PRESERVE_PATH)) - (POLICY(pfrd,DOESNT_PRESERVE_PATH)) - (POLICY(rand,DOESNT_PRESERVE_PATH)); - -const Policy * const Policy::policies = &_policies_[1]; - -#define CONST_POLICY(X) const Policy &Policy::X = Policy::policies[Policy::Enum::X] - -CONST_POLICY(invalid); -CONST_POLICY(all); -CONST_POLICY(epall); -CONST_POLICY(epff); -CONST_POLICY(eplfs); -CONST_POLICY(eplus); -CONST_POLICY(epmfs); -CONST_POLICY(eppfrd); -CONST_POLICY(eprand); -CONST_POLICY(erofs); -CONST_POLICY(ff); -CONST_POLICY(lfs); -CONST_POLICY(lus); -CONST_POLICY(mfs); -CONST_POLICY(msplfs); -CONST_POLICY(msplus); -CONST_POLICY(mspmfs); -CONST_POLICY(msppfrd); -CONST_POLICY(newest); -CONST_POLICY(pfrd); -CONST_POLICY(rand); - -const Policy& -Policy::find(const std::string &str) -{ - for(int i = Enum::BEGIN; i != Enum::END; ++i) - { - if(policies[i] == str) - return policies[i]; - } - - return invalid; -} - -const Policy& -Policy::find(const Policy::Enum::Type i) -{ - if(i >= Policy::Enum::BEGIN && - i < Policy::Enum::END) - return policies[i]; - - return invalid; -} diff --git a/src/policy.hpp b/src/policy.hpp index 1663e618..e527b94a 100644 --- a/src/policy.hpp +++ b/src/policy.hpp @@ -1,5 +1,5 @@ /* - Copyright (c) 2016, Antonio SJ Musumeci + Copyright (c) 2020, Antonio SJ Musumeci Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above @@ -16,217 +16,179 @@ #pragma once -#include "branch.hpp" -#include "category.hpp" +#include "branches.hpp" +#include "strvec.hpp" #include -#include -class Policy +namespace Policy { -public: - struct Enum + class ActionImpl { - enum Type - { - invalid = -1, - BEGIN = 0, - all = BEGIN, - epall, - epff, - eplfs, - eplus, - epmfs, - eppfrd, - eprand, - erofs, - ff, - lfs, - lus, - mfs, - msplfs, - msplus, - mspmfs, - msppfrd, - newest, - pfrd, - rand, - END - }; - - static size_t begin() { return BEGIN; } - static size_t end() { return END; } + public: + ActionImpl(const std::string &name_) + : name(name_) + { + } + + public: + std::string name; + virtual int operator()(const Branches::CPtr&,const char*,StrVec*) const = 0; }; - struct Func + class Action { - typedef std::string string; - typedef std::vector strvec; - typedef const string cstring; - typedef const uint64_t cuint64_t; - typedef const strvec cstrvec; - - typedef int (*Ptr)(Category,const Branches&,const char*,strvec *); - - template - class Base - { - public: - Base(const Policy &p_) - : func(p_._func) - {} - - Base(const Policy *p_) - : func(p_->_func) - {} - - int - operator()(const Branches &a,const char *b,strvec *c) - { - return func(T,a,b,c); - } - - int - operator()(const Branches &a,const string &b,strvec *c) - { - return func(T,a,b.c_str(),c); - } - - int - operator()(const Branches &a,const char *b,string *c) - { - int rv; - strvec v; - - rv = func(T,a,b,&v); - if(!v.empty()) - *c = v[0]; - - return rv; - } - - private: - const Ptr func; - }; - - typedef Base Action; - typedef Base Create; - typedef Base Search; - - static int invalid(Category,const Branches&,const char *,strvec*); - static int all(Category,const Branches&,const char*,strvec*); - static int epall(Category,const Branches&,const char*,strvec*); - static int epff(Category,const Branches&,const char *,strvec*); - static int eplfs(Category,const Branches&,const char *,strvec*); - static int eplus(Category,const Branches&,const char *,strvec*); - static int epmfs(Category,const Branches&,const char *,strvec*); - static int eppfrd(Category,const Branches&,const char *,strvec*); - static int eprand(Category,const Branches&,const char *,strvec*); - static int erofs(Category,const Branches&,const char *,strvec*); - static int ff(Category,const Branches&,const char *,strvec*); - static int lfs(Category,const Branches&,const char *,strvec*); - static int lus(Category,const Branches&,const char *,strvec*); - static int mfs(Category,const Branches&,const char *,strvec*); - static int msplfs(Category,const Branches&,const char *,strvec*); - static int msplus(Category,const Branches&,const char *,strvec*); - static int mspmfs(Category,const Branches&,const char *,strvec*); - static int msppfrd(Category,const Branches&,const char *,strvec*); - static int newest(Category,const Branches&,const char *,strvec*); - static int pfrd(Category,const Branches&,const char *,strvec*); - static int rand(Category,const Branches&,const char *,strvec*); + public: + Action(ActionImpl *impl_) + : impl(impl_) + {} + + Action& + operator=(ActionImpl *impl_) + { + impl = impl_; + return *this; + } + + const + std::string& + name(void) const + { + return impl->name; + } + + int + operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const + { + return (*impl)(branches_,fusepath_,paths_); + } + + int + operator()(const Branches::CPtr &branches_, + const std::string &fusepath_, + StrVec *paths_) const + { + return (*impl)(branches_,fusepath_.c_str(),paths_); + } + + private: + ActionImpl *impl; }; -private: - Enum::Type _enum; - std::string _str; - Func::Ptr _func; - bool _path_preserving; - -public: - Policy() - : _enum(invalid), - _str(invalid), - _func(invalid), - _path_preserving(false) + class CreateImpl { - } - - Policy(const Enum::Type enum_, - const std::string &str_, - const Func::Ptr func_, - const bool path_preserving_) - : _enum(enum_), - _str(str_), - _func(func_), - _path_preserving(path_preserving_) + public: + CreateImpl(const std::string &name_) + : name(name_) + { + } + + public: + std::string name; + virtual int operator()(const Branches::CPtr&,const char*,StrVec*) const = 0; + virtual bool path_preserving(void) const = 0; + }; + + class Create { - } + public: + Create(CreateImpl *impl_) + : impl(impl_) + {} + + Create& + operator=(CreateImpl *impl_) + { + impl = impl_; + return *this; + } + + const + std::string& + name(void) const + { + return impl->name; + } + + bool + path_preserving(void) const + { + return impl->path_preserving(); + } - bool - path_preserving() const + int + operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const + { + return (*impl)(branches_,fusepath_,paths_); + } + + int + operator()(const Branches::CPtr &branches_, + const std::string &fusepath_, + StrVec *paths_) const + { + return (*impl)(branches_,fusepath_.c_str(),paths_); + } + + private: + CreateImpl *impl; + }; + + class SearchImpl { - return _path_preserving; - } - -public: - operator const Enum::Type() const { return _enum; } - operator const std::string&() const { return _str; } - operator const Func::Ptr() const { return _func; } - operator const Policy*() const { return this; } - const std::string& to_string() const { return _str; } - - bool operator==(const Enum::Type enum_) const - { return _enum == enum_; } - bool operator==(const std::string &str_) const - { return _str == str_; } - bool operator==(const Func::Ptr func_) const - { return _func == func_; } - - bool operator!=(const Policy &r) const - { return _enum != r._enum; } - - bool operator<(const Policy &r) const - { return _enum < r._enum; } - -public: - static const Policy &find(const std::string&); - static const Policy &find(const Enum::Type); - -public: - - static const std::vector _policies_; - static const Policy * const policies; - - static const Policy &invalid; - static const Policy &all; - static const Policy &epall; - static const Policy &epff; - static const Policy &eplfs; - static const Policy ⩱ - static const Policy &epmfs; - static const Policy &eppfrd; - static const Policy &eprand; - static const Policy &erofs; - static const Policy &ff; - static const Policy &lfs; - static const Policy &lus; - static const Policy &mfs; - static const Policy &msplfs; - static const Policy &msplus; - static const Policy &mspmfs; - static const Policy &msppfrd; - static const Policy &newest; - static const Policy &pfrd; - static const Policy &rand; -}; - -namespace std -{ - static - inline - string - to_string(const Policy &p_) + public: + SearchImpl(const std::string &name_) + : name(name_) + { + } + + public: + std::string name; + virtual int operator()(const Branches::CPtr&,const char*,StrVec*) const = 0; + }; + + class Search { - return p_.to_string(); - } + public: + Search(SearchImpl *impl_) + : impl(impl_) + {} + + Search& + operator=(SearchImpl *impl_) + { + impl = impl_; + return *this; + } + + const + std::string& + name(void) const + { + return impl->name; + } + + int + operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const + { + return (*impl)(branches_,fusepath_,paths_); + } + + int + operator()(const Branches::CPtr &branches_, + const std::string &fusepath_, + StrVec *paths_) const + { + return (*impl)(branches_,fusepath_.c_str(),paths_); + } + + private: + SearchImpl *impl; + }; } diff --git a/src/policy_all.cpp b/src/policy_all.cpp index 92d86bfc..785a634b 100644 --- a/src/policy_all.cpp +++ b/src/policy_all.cpp @@ -19,43 +19,39 @@ #include "fs_info.hpp" #include "fs_path.hpp" #include "policy.hpp" +#include "policies.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "strvec.hpp" #include -#include using std::string; -using std::vector; namespace all { static int - create(const BranchVec &branches_, - vector *paths_) + create(const Branches::CPtr &branches_, + StrVec *paths_) { int rv; int error; fs::info_t info; - const Branch *branch; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); - paths_->push_back(branch->path); + paths_->push_back(branch.path); } if(paths_->empty()) @@ -63,26 +59,28 @@ namespace all return 0; } +} - static - int - create(const Branches &branches_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return all::create(branches_.vec,paths_); - } +int +Policy::All::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::epall(branches_,fusepath_,paths_); } int -Policy::Func::all(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::All::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return all::create(branches_,paths_); + return ::all::create(branches_,paths_); +} - return Policy::Func::epall(type_,branches_,fusepath_,paths_); +int +Policy::All::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::epall(branches_,fusepath_,paths_); } diff --git a/src/policy_all.hpp b/src/policy_all.hpp new file mode 100644 index 00000000..8f5b67c8 --- /dev/null +++ b/src/policy_all.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace All + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("all") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("all") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("all") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_cache.cpp b/src/policy_cache.cpp index 08645eab..590ea54f 100644 --- a/src/policy_cache.cpp +++ b/src/policy_cache.cpp @@ -28,7 +28,7 @@ namespace l PolicyCache::Value::Value() : time(0), - path() + paths() { } @@ -91,18 +91,17 @@ PolicyCache::clear(void) } int -PolicyCache::operator()(Policy::Func::Search &func_, +PolicyCache::operator()(const Policy::Search &policy_, const Branches &branches_, const char *fusepath_, - std::string *branch_) + StrVec *paths_) { int rv; Value *v; uint64_t now; - string branch; if(timeout == 0) - return func_(branches_,fusepath_,branch_); + return policy_(branches_,fusepath_,paths_); now = l::get_time(); @@ -112,16 +111,19 @@ PolicyCache::operator()(Policy::Func::Search &func_, if((now - v->time) >= timeout) { pthread_mutex_unlock(&_lock); - rv = func_(branches_,fusepath_,&branch); + + rv = policy_(branches_,fusepath_,paths_); if(rv == -1) return -1; pthread_mutex_lock(&_lock); - v->time = now; - v->path = branch; + v->time = now; + v->paths = *paths_; + } + else + { + *paths_ = v->paths; } - - *branch_ = v->path; pthread_mutex_unlock(&_lock); diff --git a/src/policy_cache.hpp b/src/policy_cache.hpp index dc7218d9..6cdb9280 100644 --- a/src/policy_cache.hpp +++ b/src/policy_cache.hpp @@ -19,12 +19,14 @@ #pragma once #include "policy.hpp" +#include "strvec.hpp" +#include #include #include #include -#include + class PolicyCache { @@ -33,8 +35,8 @@ public: { Value(); - uint64_t time; - std::string path; + uint64_t time; + StrVec paths; }; public: @@ -46,10 +48,10 @@ public: void clear(void); public: - int operator()(Policy::Func::Search &func, + int operator()(const Policy::Search &policy, const Branches &branches, const char *fusepath, - std::string *branch); + StrVec *paths); public: uint64_t timeout; diff --git a/src/policy_epall.cpp b/src/policy_epall.cpp index 47c1992e..af4b526b 100644 --- a/src/policy_epall.cpp +++ b/src/policy_epall.cpp @@ -20,46 +20,43 @@ #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" #include "policy.hpp" +#include "policy_epall.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "strvec.hpp" #include -#include using std::string; -using std::vector; + namespace epall { static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; fs::info_t info; - const Branch *branch; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); - paths_->push_back(branch->path); + paths_->push_back(branch.path); } if(paths_->empty()) @@ -70,42 +67,28 @@ namespace epall static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return epall::create(branches_.vec,fusepath_,paths_); - } - - static - int - action(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; bool readonly; - const Branch *branch; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::statvfs_cache_readonly(branch->path,&readonly); + rv = fs::statvfs_cache_readonly(branch.path,&readonly); if(rv == -1) error_and_continue(error,ENOENT); if(readonly) error_and_continue(error,EROFS); - paths_->push_back(branch->path); + paths_->push_back(branch.path); } if(paths_->empty()) @@ -116,31 +99,16 @@ namespace epall static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { - rwlock::ReadGuard guard(branches_.lock); - - return epall::action(branches_.vec,fusepath_,paths_); - } - - static - int - search(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) - { - const Branch *branch; - - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) continue; - paths_->push_back(branch->path); + paths_->push_back(branch.path); } if(paths_->empty()) @@ -148,33 +116,28 @@ namespace epall return 0; } +} - static - int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); +int +Policy::EPAll::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::epall::action(branches_,fusepath_,paths_); +} - return epall::search(branches_.vec,fusepath_,paths_); - } +int +Policy::EPAll::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::epall::create(branches_,fusepath_,paths_); } int -Policy::Func::epall(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPAll::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return epall::create(branches_,fusepath_,paths_); - case Category::ACTION: - return epall::action(branches_,fusepath_,paths_); - case Category::SEARCH: - default: - return epall::search(branches_,fusepath_,paths_); - } + return ::epall::search(branches_,fusepath_,paths_); } diff --git a/src/policy_epall.hpp b/src/policy_epall.hpp new file mode 100644 index 00000000..1b665671 --- /dev/null +++ b/src/policy_epall.hpp @@ -0,0 +1,64 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPAll + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("epall") + { + } + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("epall") + { + } + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("epall") + { + } + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_epff.cpp b/src/policy_epff.cpp index ac519c96..d2980f42 100644 --- a/src/policy_epff.cpp +++ b/src/policy_epff.cpp @@ -14,12 +14,14 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include "branches.hpp" #include "errno.hpp" #include "fs_exists.hpp" #include "fs_info.hpp" #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" #include "policy.hpp" +#include "policy_epff.hpp" #include "policy_error.hpp" #include "rwlock.hpp" @@ -33,33 +35,30 @@ namespace epff { static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; fs::info_t info; - const Branch *branch; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); - paths_->push_back(branch->path); + paths_->push_back(branch.path); return 0; } @@ -69,42 +68,28 @@ namespace epff static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return epff::create(branches_.vec,fusepath_,paths_); - } - - static - int - action(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; bool readonly; - const Branch *branch; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::statvfs_cache_readonly(branch->path,&readonly); + rv = fs::statvfs_cache_readonly(branch.path,&readonly); if(rv == -1) error_and_continue(error,ENOENT); if(readonly) error_and_continue(error,EROFS); - paths_->push_back(branch->path); + paths_->push_back(branch.path); return 0; } @@ -114,64 +99,44 @@ namespace epff static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return epff::action(branches_.vec,fusepath_,paths_); - } - - static - int - search(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { - const Branch *branch; - - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) continue; - paths_->push_back(branch->path); + paths_->push_back(branch.path); return 0; } return (errno=ENOENT,-1); } +} - static - int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); +int +Policy::EPFF::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::epff::action(branches_,fusepath_,paths_); +} - return epff::search(branches_.vec,fusepath_,paths_); - } +int +Policy::EPFF::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::epff::create(branches_,fusepath_,paths_); } int -Policy::Func::epff(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPFF::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return epff::create(branches_,fusepath_,paths_); - case Category::ACTION: - return epff::action(branches_,fusepath_,paths_); - case Category::SEARCH: - default: - return epff::search(branches_,fusepath_,paths_); - } + return ::epff::search(branches_,fusepath_,paths_); } diff --git a/src/policy_epff.hpp b/src/policy_epff.hpp new file mode 100644 index 00000000..0e3e5f4a --- /dev/null +++ b/src/policy_epff.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPFF + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("epff") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("epff") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("epff") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_eplfs.cpp b/src/policy_eplfs.cpp index 26164fdd..3ef4ed18 100644 --- a/src/policy_eplfs.cpp +++ b/src/policy_eplfs.cpp @@ -19,9 +19,10 @@ #include "fs_info.hpp" #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" +#include "policies.hpp" #include "policy.hpp" +#include "policy_eplfs.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" #include #include @@ -34,40 +35,37 @@ namespace eplfs { static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; uint64_t eplfs; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; eplfs = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); if(info.spaceavail > eplfs) continue; eplfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -80,40 +78,26 @@ namespace eplfs static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return eplfs::create(branches_.vec,fusepath_,paths_); - } - - static - int - action(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; uint64_t eplfs; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; eplfs = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) @@ -122,7 +106,7 @@ namespace eplfs continue; eplfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -135,43 +119,29 @@ namespace eplfs static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return eplfs::action(branches_.vec,fusepath_,paths_); - } - - static - int - search(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; uint64_t eplfs; uint64_t spaceavail; - const Branch *branch; const string *basepath; eplfs = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) continue; - rv = fs::statvfs_cache_spaceavail(branch->path,&spaceavail); + rv = fs::statvfs_cache_spaceavail(branch.path,&spaceavail); if(rv == -1) continue; if(spaceavail > eplfs) continue; eplfs = spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -181,33 +151,28 @@ namespace eplfs return 0; } +} - static - int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); +int +Policy::EPLFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::eplfs::action(branches_,fusepath_,paths_); +} - return eplfs::search(branches_.vec,fusepath_,paths_); - } +int +Policy::EPLFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::eplfs::create(branches_,fusepath_,paths_); } int -Policy::Func::eplfs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPLFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return eplfs::create(branches_,fusepath_,paths_); - case Category::ACTION: - return eplfs::action(branches_,fusepath_,paths_); - case Category::SEARCH: - default: - return eplfs::search(branches_,fusepath_,paths_); - } + return ::eplfs::search(branches_,fusepath_,paths_); } diff --git a/src/policy_eplfs.hpp b/src/policy_eplfs.hpp new file mode 100644 index 00000000..60006b06 --- /dev/null +++ b/src/policy_eplfs.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPLFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("eplfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("eplfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("eplfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_eplus.cpp b/src/policy_eplus.cpp index b8c79ab1..e5bac154 100644 --- a/src/policy_eplus.cpp +++ b/src/policy_eplus.cpp @@ -20,54 +20,50 @@ #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" #include "policy.hpp" +#include "policy_eplus.hpp" #include "policy_error.hpp" #include "rwlock.hpp" #include #include -#include using std::string; -using std::vector; namespace eplus { static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; uint64_t eplus; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; eplus = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); if(info.spaceused >= eplus) continue; eplus = info.spaceused; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -80,40 +76,26 @@ namespace eplus static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return eplus::create(branches_.vec,fusepath_,paths_); - } - - static - int - action(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; uint64_t eplus; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; eplus = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) @@ -122,7 +104,7 @@ namespace eplus continue; eplus = info.spaceused; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -135,43 +117,29 @@ namespace eplus static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return eplus::action(branches_.vec,fusepath_,paths_); - } - - static - int - search(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; uint64_t eplus; uint64_t spaceused; - const Branch *branch; const string *basepath; eplus = 0; basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) continue; - rv = fs::statvfs_cache_spaceused(branch->path,&spaceused); + rv = fs::statvfs_cache_spaceused(branch.path,&spaceused); if(rv == -1) continue; if(spaceused >= eplus) continue; eplus = spaceused; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -181,33 +149,28 @@ namespace eplus return 0; } +} - static - int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); +int +Policy::EPLUS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::eplus::action(branches_,fusepath_,paths_); +} - return eplus::search(branches_.vec,fusepath_,paths_); - } +int +Policy::EPLUS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::eplus::create(branches_,fusepath_,paths_); } int -Policy::Func::eplus(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPLUS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return eplus::create(branches_,fusepath_,paths_); - case Category::ACTION: - return eplus::action(branches_,fusepath_,paths_); - case Category::SEARCH: - default: - return eplus::search(branches_,fusepath_,paths_); - } + return ::eplus::search(branches_,fusepath_,paths_); } diff --git a/src/policy_eplus.hpp b/src/policy_eplus.hpp new file mode 100644 index 00000000..ca253f7e --- /dev/null +++ b/src/policy_eplus.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPLUS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("eplus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("eplus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("eplus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_epmfs.cpp b/src/policy_epmfs.cpp index b4e24919..7f44c032 100644 --- a/src/policy_epmfs.cpp +++ b/src/policy_epmfs.cpp @@ -20,54 +20,51 @@ #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" #include "policy.hpp" +#include "policy_epmfs.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" #include #include -#include + using std::string; -using std::vector; + namespace epmfs { static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; uint64_t epmfs; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; epmfs = std::numeric_limits::min(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); if(info.spaceavail < epmfs) continue; epmfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -80,40 +77,26 @@ namespace epmfs static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return epmfs::create(branches_.vec,fusepath_,paths_); - } - - static - int - action(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; uint64_t epmfs; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; epmfs = std::numeric_limits::min(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) @@ -122,7 +105,7 @@ namespace epmfs continue; epmfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -135,43 +118,29 @@ namespace epmfs static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return epmfs::action(branches_.vec,fusepath_,paths_); - } - - static - int - search(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; uint64_t epmfs; uint64_t spaceavail; - const Branch *branch; const string *basepath; epmfs = 0; basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) continue; - rv = fs::statvfs_cache_spaceavail(branch->path,&spaceavail); + rv = fs::statvfs_cache_spaceavail(branch.path,&spaceavail); if(rv == -1) continue; if(spaceavail < epmfs) continue; epmfs = spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -181,33 +150,28 @@ namespace epmfs return 0; } +} - static - int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); +int +Policy::EPMFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::epmfs::action(branches_,fusepath_,paths_); +} - return epmfs::search(branches_.vec,fusepath_,paths_); - } +int +Policy::EPMFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::epmfs::create(branches_,fusepath_,paths_); } int -Policy::Func::epmfs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPMFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return epmfs::create(branches_,fusepath_,paths_); - case Category::ACTION: - return epmfs::action(branches_,fusepath_,paths_); - case Category::SEARCH: - default: - return epmfs::search(branches_,fusepath_,paths_); - } + return ::epmfs::search(branches_,fusepath_,paths_); } diff --git a/src/policy_epmfs.hpp b/src/policy_epmfs.hpp new file mode 100644 index 00000000..e0b8382a --- /dev/null +++ b/src/policy_epmfs.hpp @@ -0,0 +1,64 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPMFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("epmfs") + { + } + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("epmfs") + { + } + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("epmfs") + { + } + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_eppfrd.cpp b/src/policy_eppfrd.cpp index 2c38efc2..d2ba449c 100644 --- a/src/policy_eppfrd.cpp +++ b/src/policy_eppfrd.cpp @@ -20,9 +20,11 @@ #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" #include "policy.hpp" +#include "policy_eppfrd.hpp" #include "policy_error.hpp" #include "rnd.hpp" #include "rwlock.hpp" +#include "strvec.hpp" #include #include @@ -42,39 +44,36 @@ namespace eppfrd { static int - get_branchinfo_create(const BranchVec &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) + get_branchinfo_create(const Branches::CPtr &branches_, + const char *fusepath_, + BranchInfoVec *branchinfo_, + uint64_t *sum_) { int rv; int error; BranchInfo bi; fs::info_t info; - const Branch *branch; *sum_ = 0; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i < ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) - error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + if(!fs::exists(branch.path,fusepath_)) + error_and_continue(error,ENOENT); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); *sum_ += info.spaceavail; bi.spaceavail = info.spaceavail; - bi.basepath = &branch->path; + bi.basepath = &branch.path; branchinfo_->push_back(bi); } @@ -83,42 +82,25 @@ namespace eppfrd static int - get_branchinfo_create(const Branches &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) - { - rwlock::ReadGuard guard(branches_.lock); - - branchinfo_->reserve(branches_.vec.size()); - - return eppfrd::get_branchinfo_create(branches_.vec,fusepath_,branchinfo_,sum_); - } - - static - int - get_branchinfo_action(const BranchVec &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) + get_branchinfo_action(const Branches::CPtr &branches_, + const char *fusepath_, + BranchInfoVec *branchinfo_, + uint64_t *sum_) { int rv; int error; BranchInfo bi; fs::info_t info; - const Branch *branch; *sum_ = 0; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i < ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) @@ -127,7 +109,7 @@ namespace eppfrd *sum_ += info.spaceavail; bi.spaceavail = info.spaceavail; - bi.basepath = &branch->path; + bi.basepath = &branch.path; branchinfo_->push_back(bi); } @@ -136,65 +118,34 @@ namespace eppfrd static int - get_branchinfo_action(const Branches &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) - { - rwlock::ReadGuard guard(branches_.lock); - - branchinfo_->reserve(branches_.vec.size()); - - return eppfrd::get_branchinfo_action(branches_.vec,fusepath_,branchinfo_,sum_); - } - - static - int - get_branchinfo_search(const BranchVec &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) + get_branchinfo_search(const Branches::CPtr &branches_, + const char *fusepath_, + BranchInfoVec *branchinfo_, + uint64_t *sum_) { int rv; BranchInfo bi; uint64_t spaceavail; - const Branch *branch; *sum_ = 0; - for(size_t i = 0, ei = branches_.size(); i < ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) continue; - rv = fs::statvfs_cache_spaceavail(branch->path,&spaceavail); + rv = fs::statvfs_cache_spaceavail(branch.path,&spaceavail); if(rv == -1) continue; *sum_ += spaceavail; bi.spaceavail = spaceavail; - bi.basepath = &branch->path; + bi.basepath = &branch.path; branchinfo_->push_back(bi); } return ENOENT; } - static - int - get_branchinfo_search(const Branches &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) - { - rwlock::ReadGuard guard(branches_.lock); - - branchinfo_->reserve(branches_.vec.size()); - - return eppfrd::get_branchinfo_search(branches_.vec,fusepath_,branchinfo_,sum_); - } - static const string* @@ -221,9 +172,9 @@ namespace eppfrd static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; uint64_t sum; @@ -242,9 +193,9 @@ namespace eppfrd static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; uint64_t sum; @@ -263,9 +214,9 @@ namespace eppfrd static int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; uint64_t sum; @@ -284,19 +235,25 @@ namespace eppfrd } int -Policy::Func::eppfrd(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPPFRD::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::eppfrd::action(branches_,fusepath_,paths_); +} + +int +Policy::EPPFRD::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::eppfrd::create(branches_,fusepath_,paths_); +} + +int +Policy::EPPFRD::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return eppfrd::create(branches_,fusepath_,paths_); - case Category::ACTION: - return eppfrd::action(branches_,fusepath_,paths_); - default: - case Category::SEARCH: - return eppfrd::search(branches_,fusepath_,paths_); - } + return ::eppfrd::search(branches_,fusepath_,paths_); } diff --git a/src/policy_eppfrd.hpp b/src/policy_eppfrd.hpp new file mode 100644 index 00000000..f3b6cbc3 --- /dev/null +++ b/src/policy_eppfrd.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPPFRD + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("eppfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("eppfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("eppfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_eprand.cpp b/src/policy_eprand.cpp index 65dd9ffa..248249c8 100644 --- a/src/policy_eprand.cpp +++ b/src/policy_eprand.cpp @@ -15,24 +15,54 @@ */ #include "errno.hpp" +#include "policies.hpp" #include "policy.hpp" +#include "policy_eprand.hpp" #include -#include -#include -using std::string; -using std::vector; +int +Policy::EPRand::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + int rv; + + rv = Policies::Action::epall(branches_,fusepath_,paths_); + if(rv == 0) + { + std::random_shuffle(paths_->begin(),paths_->end()); + paths_->resize(1); + } + + return rv; +} + +int +Policy::EPRand::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + int rv; + + rv = Policies::Create::epall(branches_,fusepath_,paths_); + if(rv == 0) + { + std::random_shuffle(paths_->begin(),paths_->end()); + paths_->resize(1); + } + + return rv; +} int -Policy::Func::eprand(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::EPRand::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { int rv; - rv = Policy::Func::epall(type_,branches_,fusepath_,paths_); + rv = Policies::Search::epall(branches_,fusepath_,paths_); if(rv == 0) { std::random_shuffle(paths_->begin(),paths_->end()); diff --git a/src/policy_eprand.hpp b/src/policy_eprand.hpp new file mode 100644 index 00000000..f0a62b4f --- /dev/null +++ b/src/policy_eprand.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace EPRand + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("eprand") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("eprand") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("eprand") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_erofs.cpp b/src/policy_erofs.cpp index c7ee6682..27376be3 100644 --- a/src/policy_erofs.cpp +++ b/src/policy_erofs.cpp @@ -16,18 +16,33 @@ #include "errno.hpp" #include "policy.hpp" +#include "policy_erofs.hpp" #include -#include using std::string; -using std::vector; + + +int +Policy::ERoFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return (errno=EROFS,-1); +} + +int +Policy::ERoFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return (errno=EROFS,-1); +} int -Policy::Func::erofs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths) +Policy::ERoFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { return (errno=EROFS,-1); } diff --git a/src/policy_erofs.hpp b/src/policy_erofs.hpp new file mode 100644 index 00000000..f17f4053 --- /dev/null +++ b/src/policy_erofs.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace ERoFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("erofs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("erofs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("erofs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_ff.cpp b/src/policy_ff.cpp index 72a8f1e2..4cf00b32 100644 --- a/src/policy_ff.cpp +++ b/src/policy_ff.cpp @@ -18,70 +18,68 @@ #include "fs_exists.hpp" #include "fs_info.hpp" #include "fs_path.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "policy_ff.hpp" #include -#include using std::string; -using std::vector; namespace ff { static int - create(const BranchVec &branches_, - vector *paths_) + create(const Branches::CPtr &branches_, + StrVec *paths_) { int rv; int error; fs::info_t info; - const Branch *branch; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); - paths_->push_back(branch->path); + paths_->push_back(branch.path); return 0; } return (errno=error,-1); } +} - static - int - create(const Branches &branches_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return ff::create(branches_.vec,paths_); - } +int +Policy::FF::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::epff(branches_,fusepath_,paths_); } int -Policy::Func::ff(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::FF::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return ff::create(branches_,paths_); + return ::ff::create(branches_,paths_); +} - return Policy::Func::epff(type_,branches_,fusepath_,paths_); +int +Policy::FF::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::epff(branches_,fusepath_,paths_); } diff --git a/src/policy_ff.hpp b/src/policy_ff.hpp new file mode 100644 index 00000000..1b0db87f --- /dev/null +++ b/src/policy_ff.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace FF + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("ff") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("ff") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving(void) const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("ff") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_lfs.cpp b/src/policy_lfs.cpp index 05000c18..ace2a1bf 100644 --- a/src/policy_lfs.cpp +++ b/src/policy_lfs.cpp @@ -18,23 +18,23 @@ #include "fs_exists.hpp" #include "fs_info.hpp" #include "fs_path.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "policy_lfs.hpp" +#include "strvec.hpp" #include #include -#include using std::string; -using std::vector; namespace lfs { static int - create(const BranchVec &branches_, - vector *paths_) + create(const Branches::CPtr &branches_, + StrVec *paths_) { int rv; int error; @@ -46,24 +46,22 @@ namespace lfs error = ENOENT; lfs = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); if(info.spaceavail > lfs) continue; lfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -73,26 +71,28 @@ namespace lfs return 0; } +} - static - int - create(const Branches &branches_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return lfs::create(branches_.vec,paths_); - } +int +Policy::LFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::eplfs(branches_,fusepath_,paths_); } int -Policy::Func::lfs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::LFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return lfs::create(branches_,paths_); + return ::lfs::create(branches_,paths_); +} - return Policy::Func::eplfs(type_,branches_,fusepath_,paths_); +int +Policy::LFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::eplfs(branches_,fusepath_,paths_); } diff --git a/src/policy_lfs.hpp b/src/policy_lfs.hpp new file mode 100644 index 00000000..e0b456ef --- /dev/null +++ b/src/policy_lfs.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace LFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("lfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("lfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("lfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_lus.cpp b/src/policy_lus.cpp index 3576afad..c23570aa 100644 --- a/src/policy_lus.cpp +++ b/src/policy_lus.cpp @@ -18,9 +18,11 @@ #include "fs_exists.hpp" #include "fs_info.hpp" #include "fs_path.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "policy_lus.hpp" +#include "strvec.hpp" #include #include @@ -33,37 +35,34 @@ namespace lus { static int - create(const BranchVec &branches_, - vector *paths_) + create(const Branches::CPtr &branches_, + StrVec *paths_) { int rv; int error; uint64_t lus; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; lus = std::numeric_limits::max(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); if(info.spaceused >= lus) continue; lus = info.spaceused; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -73,26 +72,28 @@ namespace lus return 0; } +} - static - int - create(const Branches &branches_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return lus::create(branches_.vec,paths_); - } +int +Policy::LUS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::eplus(branches_,fusepath_,paths_); } int -Policy::Func::lus(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::LUS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return lus::create(branches_,paths_); + return ::lus::create(branches_,paths_); +} - return Policy::Func::eplus(type_,branches_,fusepath_,paths_); +int +Policy::LUS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::eplus(branches_,fusepath_,paths_); } diff --git a/src/policy_lus.hpp b/src/policy_lus.hpp new file mode 100644 index 00000000..b6c07717 --- /dev/null +++ b/src/policy_lus.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace LUS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("lus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("lus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("lus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_mfs.cpp b/src/policy_mfs.cpp index ed766d95..489e0cd9 100644 --- a/src/policy_mfs.cpp +++ b/src/policy_mfs.cpp @@ -18,51 +18,46 @@ #include "fs_exists.hpp" #include "fs_info.hpp" #include "fs_path.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" #include -#include using std::string; -using std::vector; namespace mfs { static int - create(const BranchVec &branches_, - vector *paths_) + create(const Branches::CPtr &branches_, + StrVec *paths_) { int rv; int error; uint64_t mfs; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; mfs = 0; basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); if(info.spaceavail < mfs) continue; mfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -72,26 +67,28 @@ namespace mfs return 0; } +} - static - int - create(const Branches &branches_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return mfs::create(branches_.vec,paths_); - } +int +Policy::MFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::mfs(branches_,fusepath_,paths_); } int -Policy::Func::mfs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::MFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return mfs::create(branches_,paths_); + return ::mfs::create(branches_,paths_); +} - return Policy::Func::epmfs(type_,branches_,fusepath_,paths_); +int +Policy::MFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::epmfs(branches_,fusepath_,paths_); } diff --git a/src/policy_mfs.hpp b/src/policy_mfs.hpp new file mode 100644 index 00000000..9247f77a --- /dev/null +++ b/src/policy_mfs.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace MFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("mfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("mfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("mfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_msplfs.cpp b/src/policy_msplfs.cpp index 3d6a6f61..27039a62 100644 --- a/src/policy_msplfs.cpp +++ b/src/policy_msplfs.cpp @@ -19,16 +19,16 @@ #include "fs_info.hpp" #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "policy_msplfs.hpp" +#include "strvec.hpp" #include #include -#include using std::string; -using std::vector; namespace msplfs @@ -36,38 +36,35 @@ namespace msplfs static const string* - create_1(const BranchVec &branches_, - const string &fusepath_, - int *err_) + create_1(const Branches::CPtr &branches_, + const string &fusepath_, + int *err_) { int rv; uint64_t lfs; fs::info_t info; - const Branch *branch; const string *basepath; basepath = NULL; lfs = std::numeric_limits::max(); - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(*err_,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(*err_,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(*err_,ENOENT); if(info.readonly) error_and_continue(*err_,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(*err_,ENOSPC); if(info.spaceavail > lfs) continue; lfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } return basepath; @@ -75,9 +72,9 @@ namespace msplfs static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; string fusepath; @@ -102,27 +99,28 @@ namespace msplfs return 0; } +} - static - int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return msplfs::create(branches_.vec,fusepath_,paths_); - } +int +Policy::MSPLFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::eplfs(branches_,fusepath_,paths_); } int -Policy::Func::msplfs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::MSPLFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return msplfs::create(branches_,fusepath_,paths_); + return ::msplfs::create(branches_,fusepath_,paths_); +} - return Policy::Func::eplfs(type_,branches_,fusepath_,paths_); +int +Policy::MSPLFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::eplfs(branches_,fusepath_,paths_); } diff --git a/src/policy_msplfs.hpp b/src/policy_msplfs.hpp new file mode 100644 index 00000000..7e0a8d3e --- /dev/null +++ b/src/policy_msplfs.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace MSPLFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("msplfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("msplfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("msplfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_msplus.cpp b/src/policy_msplus.cpp index 28278fc2..e6e0cb87 100644 --- a/src/policy_msplus.cpp +++ b/src/policy_msplus.cpp @@ -19,55 +19,50 @@ #include "fs_info.hpp" #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "policy_msplus.hpp" #include #include -#include using std::string; -using std::vector; - namespace msplus { static const string* - create_1(const BranchVec &branches_, - const string &fusepath_, - int *err_) + create_1(const Branches::CPtr &branches_, + const string &fusepath_, + int *err_) { int rv; uint64_t lus; fs::info_t info; - const Branch *branch; const string *basepath; basepath = NULL; lus = std::numeric_limits::max(); - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(*err_,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(*err_,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(*err_,ENOENT); if(info.readonly) error_and_continue(*err_,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(*err_,ENOSPC); if(info.spaceused >= lus) continue; lus = info.spaceused;; - basepath = &branch->path; + basepath = &branch.path; } return basepath; @@ -75,9 +70,9 @@ namespace msplus static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; string fusepath; @@ -102,27 +97,28 @@ namespace msplus return 0; } +} - static - int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return msplus::create(branches_.vec,fusepath_,paths_); - } +int +Policy::MSPLUS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::eplus(branches_,fusepath_,paths_); } int -Policy::Func::msplus(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::MSPLUS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return msplus::create(branches_,fusepath_,paths_); + return ::msplus::create(branches_,fusepath_,paths_); +} - return Policy::Func::eplus(type_,branches_,fusepath_,paths_); +int +Policy::MSPLUS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::eplus(branches_,fusepath_,paths_); } diff --git a/src/policy_msplus.hpp b/src/policy_msplus.hpp new file mode 100644 index 00000000..5ccf06eb --- /dev/null +++ b/src/policy_msplus.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace MSPLUS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("msplus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("msplus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("msplus") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_mspmfs.cpp b/src/policy_mspmfs.cpp index edc8e9b6..ec00f496 100644 --- a/src/policy_mspmfs.cpp +++ b/src/policy_mspmfs.cpp @@ -19,9 +19,10 @@ #include "fs_info.hpp" #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" -#include "rwlock.hpp" +#include "policy_mspmfs.hpp" #include #include @@ -36,38 +37,35 @@ namespace mspmfs static const string* - create_1(const BranchVec &branches_, - const string &fusepath_, - int *err_) + create_1(const Branches::CPtr &branches_, + const string &fusepath_, + int *err_) { int rv; uint64_t mfs; fs::info_t info; - const Branch *branch; const string *basepath; basepath = NULL; mfs = std::numeric_limits::min(); - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(const auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(*err_,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(*err_,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(*err_,ENOENT); if(info.readonly) error_and_continue(*err_,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(*err_,ENOSPC); if(info.spaceavail < mfs) continue; mfs = info.spaceavail; - basepath = &branch->path; + basepath = &branch.path; } return basepath; @@ -75,9 +73,9 @@ namespace mspmfs static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; string fusepath; @@ -102,27 +100,28 @@ namespace mspmfs return 0; } +} - static - int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return mspmfs::create(branches_.vec,fusepath_,paths_); - } +int +Policy::MSPMFS::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Action::epmfs(branches_,fusepath_,paths_); } int -Policy::Func::mspmfs(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::MSPMFS::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return mspmfs::create(branches_,fusepath_,paths_); + return ::mspmfs::create(branches_,fusepath_,paths_); +} - return Policy::Func::epmfs(type_,branches_,fusepath_,paths_); +int +Policy::MSPMFS::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::epmfs(branches_,fusepath_,paths_); } diff --git a/src/policy_mspmfs.hpp b/src/policy_mspmfs.hpp new file mode 100644 index 00000000..3123e952 --- /dev/null +++ b/src/policy_mspmfs.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace MSPMFS + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("mspmfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("mspmfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return true; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("mspmfs") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_msppfrd.cpp b/src/policy_msppfrd.cpp index dd4f8931..baa1bd95 100644 --- a/src/policy_msppfrd.cpp +++ b/src/policy_msppfrd.cpp @@ -22,9 +22,10 @@ #include "fs_path.hpp" #include "fs_statvfs_cache.hpp" #include "policy.hpp" +#include "policy_msppfrd.hpp" +#include "policies.hpp" #include "policy_error.hpp" #include "rnd.hpp" -#include "rwlock.hpp" #include #include @@ -44,39 +45,36 @@ namespace msppfrd { static int - create_1(const BranchVec &branches_, - const string &fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) + create_1(const Branches::CPtr &branches_, + const string &fusepath_, + BranchInfoVec *branchinfo_, + uint64_t *sum_) { int rv; int error; BranchInfo bi; fs::info_t info; - const Branch *branch; *sum_ = 0; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i < ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_)) + if(!fs::exists(branch.path,fusepath_)) error_and_continue(error,ENOENT); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); *sum_ += info.spaceavail; bi.spaceavail = info.spaceavail; - bi.basepath = &branch->path; + bi.basepath = &branch.path; branchinfo_->push_back(bi); } @@ -85,24 +83,10 @@ namespace msppfrd static int - create_1(const Branches &branches_, - const string &fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) - { - rwlock::ReadGuard guard(branches_.lock); - - branchinfo_->reserve(branches_.vec.size()); - - return msppfrd::create_1(branches_.vec,fusepath_,branchinfo_,sum_); - } - - static - int - get_branchinfo(const Branches &branches_, - const char *fusepath_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) + get_branchinfo(const Branches::CPtr &branches_, + const char *fusepath_, + BranchInfoVec *branchinfo_, + uint64_t *sum_) { int error; string fusepath; @@ -147,9 +131,9 @@ namespace msppfrd static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; uint64_t sum; @@ -168,13 +152,25 @@ namespace msppfrd } int -Policy::Func::msppfrd(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::MSPPFRD::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return msppfrd::create(branches_,fusepath_,paths_); + return Policies::Action::eppfrd(branches_,fusepath_,paths_); +} - return Policy::Func::eppfrd(type_,branches_,fusepath_,paths_); +int +Policy::MSPPFRD::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::msppfrd::create(branches_,fusepath_,paths_); +} + +int +Policy::MSPPFRD::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::eppfrd(branches_,fusepath_,paths_); } diff --git a/src/policy_msppfrd.hpp b/src/policy_msppfrd.hpp new file mode 100644 index 00000000..d357d033 --- /dev/null +++ b/src/policy_msppfrd.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace MSPPFRD + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("msppfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("msppfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return true; }; + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("msppfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_newest.cpp b/src/policy_newest.cpp index 520cc1c9..f65e0f66 100644 --- a/src/policy_newest.cpp +++ b/src/policy_newest.cpp @@ -21,56 +21,52 @@ #include "fs_statvfs_cache.hpp" #include "policy.hpp" #include "policy_error.hpp" +#include "policy_newest.hpp" #include "rwlock.hpp" #include -#include #include #include using std::string; -using std::vector; namespace newest { static int - create(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; time_t newest; struct stat st; fs::info_t info; - const Branch *branch; const string *basepath; error = ENOENT; newest = std::numeric_limits::min(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_,&st)) + if(!fs::exists(branch.path,fusepath_,&st)) error_and_continue(error,ENOENT); if(st.st_mtime < newest) continue; - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); newest = st.st_mtime; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -83,50 +79,36 @@ namespace newest static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return newest::create(branches_.vec,fusepath_,paths_); - } - - static - int - action(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + action(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int rv; int error; bool readonly; time_t newest; struct stat st; - const Branch *branch; const string *basepath; error = ENOENT; newest = std::numeric_limits::min(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro()) + if(branch.ro()) error_and_continue(error,EROFS); - if(!fs::exists(branch->path,fusepath_,&st)) + if(!fs::exists(branch.path,fusepath_,&st)) error_and_continue(error,ENOENT); if(st.st_mtime < newest) continue; - rv = fs::statvfs_cache_readonly(branch->path,&readonly); + rv = fs::statvfs_cache_readonly(branch.path,&readonly); if(rv == -1) error_and_continue(error,ENOENT); if(readonly) error_and_continue(error,EROFS); newest = st.st_mtime; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -139,39 +121,25 @@ namespace newest static int - action(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); - - return newest::action(branches_.vec,fusepath_,paths_); - } - - static - int - search(const BranchVec &branches_, - const char *fusepath_, - vector *paths_) + search(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { time_t newest; struct stat st; - const Branch *branch; const string *basepath; newest = std::numeric_limits::min(); basepath = NULL; - for(size_t i = 0, ei = branches_.size(); i != ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(!fs::exists(branch->path,fusepath_,&st)) + if(!fs::exists(branch.path,fusepath_,&st)) continue; if(st.st_mtime < newest) continue; newest = st.st_mtime; - basepath = &branch->path; + basepath = &branch.path; } if(basepath == NULL) @@ -181,33 +149,28 @@ namespace newest return 0; } +} - static - int - search(const Branches &branches_, - const char *fusepath_, - vector *paths_) - { - rwlock::ReadGuard guard(branches_.lock); +int +Policy::Newest::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::newest::action(branches_,fusepath_,paths_); +} - return newest::search(branches_.vec,fusepath_,paths_); - } +int +Policy::Newest::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::newest::create(branches_,fusepath_,paths_); } int -Policy::Func::newest(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::Newest::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - switch(type_) - { - case Category::CREATE: - return newest::create(branches_,fusepath_,paths_); - case Category::ACTION: - return newest::action(branches_,fusepath_,paths_); - case Category::SEARCH: - default: - return newest::search(branches_,fusepath_,paths_); - } + return ::newest::search(branches_,fusepath_,paths_); } diff --git a/src/policy_newest.hpp b/src/policy_newest.hpp new file mode 100644 index 00000000..c4baf298 --- /dev/null +++ b/src/policy_newest.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace Newest + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("newest") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("newest") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("newest") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_pfrd.cpp b/src/policy_pfrd.cpp index ab22e278..2862d3ca 100644 --- a/src/policy_pfrd.cpp +++ b/src/policy_pfrd.cpp @@ -17,10 +17,12 @@ #include "errno.hpp" #include "fs_info.hpp" #include "fs_path.hpp" +#include "policies.hpp" #include "policy.hpp" #include "policy_error.hpp" +#include "policy_pfrd.hpp" #include "rnd.hpp" -#include "rwlock.hpp" +#include "strvec.hpp" #include #include @@ -40,55 +42,39 @@ namespace pfrd { static int - get_branchinfo(const BranchVec &branches_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) + get_branchinfo(const Branches::CPtr &branches_, + BranchInfoVec *branchinfo_, + uint64_t *sum_) { int rv; int error; BranchInfo bi; fs::info_t info; - const Branch *branch; *sum_ = 0; error = ENOENT; - for(size_t i = 0, ei = branches_.size(); i < ei; i++) + for(auto &branch : *branches_) { - branch = &branches_[i]; - - if(branch->ro_or_nc()) + if(branch.ro_or_nc()) error_and_continue(error,EROFS); - rv = fs::info(branch->path,&info); + rv = fs::info(branch.path,&info); if(rv == -1) error_and_continue(error,ENOENT); if(info.readonly) error_and_continue(error,EROFS); - if(info.spaceavail < branch->minfreespace()) + if(info.spaceavail < branch.minfreespace()) error_and_continue(error,ENOSPC); *sum_ += info.spaceavail; bi.spaceavail = info.spaceavail; - bi.basepath = &branch->path; + bi.basepath = &branch.path; branchinfo_->push_back(bi); } return error; } - static - int - get_branchinfo(const Branches &branches_, - BranchInfoVec *branchinfo_, - uint64_t *sum_) - { - rwlock::ReadGuard guard(branches_.lock); - - branchinfo_->reserve(branches_.vec.size()); - - return pfrd::get_branchinfo(branches_.vec,branchinfo_,sum_); - } - static const string* @@ -115,9 +101,9 @@ namespace pfrd static int - create(const Branches &branches_, - const char *fusepath_, - vector *paths_) + create(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) { int error; uint64_t sum; @@ -136,13 +122,25 @@ namespace pfrd } int -Policy::Func::pfrd(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::PFRD::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { - if(type_ == Category::CREATE) - return pfrd::create(branches_,fusepath_,paths_); + return Policies::Action::eppfrd(branches_,fusepath_,paths_); +} - return Policy::Func::eppfrd(type_,branches_,fusepath_,paths_); +int +Policy::PFRD::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return ::pfrd::create(branches_,fusepath_,paths_); +} + +int +Policy::PFRD::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + return Policies::Search::eppfrd(branches_,fusepath_,paths_); } diff --git a/src/policy_pfrd.hpp b/src/policy_pfrd.hpp new file mode 100644 index 00000000..86b3c11c --- /dev/null +++ b/src/policy_pfrd.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace PFRD + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("pfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("pfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("pfrd") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/policy_rand.cpp b/src/policy_rand.cpp index 160d39e5..9a74f017 100644 --- a/src/policy_rand.cpp +++ b/src/policy_rand.cpp @@ -16,23 +16,53 @@ #include "errno.hpp" #include "policy.hpp" +#include "policy_rand.hpp" +#include "policies.hpp" #include -#include -#include -using std::string; -using std::vector; +int +Policy::Rand::Action::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + int rv; + + rv = Policies::Action::all(branches_,fusepath_,paths_); + if(rv == 0) + { + std::random_shuffle(paths_->begin(),paths_->end()); + paths_->resize(1); + } + + return rv; +} + +int +Policy::Rand::Create::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const +{ + int rv; + + rv = Policies::Create::all(branches_,fusepath_,paths_); + if(rv == 0) + { + std::random_shuffle(paths_->begin(),paths_->end()); + paths_->resize(1); + } + + return rv; +} int -Policy::Func::rand(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) +Policy::Rand::Search::operator()(const Branches::CPtr &branches_, + const char *fusepath_, + StrVec *paths_) const { int rv; - rv = Policy::Func::all(type_,branches_,fusepath_,paths_); + rv = Policies::Search::all(branches_,fusepath_,paths_); if(rv == 0) { std::random_shuffle(paths_->begin(),paths_->end()); diff --git a/src/policy_rand.hpp b/src/policy_rand.hpp new file mode 100644 index 00000000..7d435433 --- /dev/null +++ b/src/policy_rand.hpp @@ -0,0 +1,61 @@ +/* + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted, provided that the above + copyright notice and this permission notice appear in all copies. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +*/ + +#pragma once + +#include "policy.hpp" + +namespace Policy +{ + namespace Rand + { + class Action final : public Policy::ActionImpl + { + public: + Action() + : Policy::ActionImpl("rand") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + + class Create final : public Policy::CreateImpl + { + public: + Create() + : Policy::CreateImpl("rand") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + bool path_preserving() const final { return false; } + }; + + class Search final : public Policy::SearchImpl + { + public: + Search() + : Policy::SearchImpl("rand") + {} + + public: + int operator()(const Branches::CPtr&,const char*,StrVec*) const final; + }; + } +} diff --git a/src/rnd.cpp b/src/rnd.cpp index 5eab6ce1..c30f3a74 100644 --- a/src/rnd.cpp +++ b/src/rnd.cpp @@ -20,9 +20,11 @@ #include "wyhash.h" -#include +#include + #include + static uint64_t G_SEED; static RND G_RND; diff --git a/src/rnd.hpp b/src/rnd.hpp index dd9b6648..e9b300d0 100644 --- a/src/rnd.hpp +++ b/src/rnd.hpp @@ -18,7 +18,8 @@ #pragma once -#include +#include + class RND { diff --git a/src/statvfs_util.hpp b/src/statvfs_util.hpp index 0447d181..a1e83dd1 100644 --- a/src/statvfs_util.hpp +++ b/src/statvfs_util.hpp @@ -16,11 +16,12 @@ #pragma once +#include #include -#include #include + namespace StatVFS { static diff --git a/src/str.cpp b/src/str.cpp index de07ff2c..66d05183 100644 --- a/src/str.cpp +++ b/src/str.cpp @@ -14,15 +14,17 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include +#include #include #include -#include #include +using std::istringstream; +using std::set; using std::string; using std::vector; -using std::istringstream; namespace str { @@ -105,6 +107,25 @@ namespace str return str::join(vec_,0,sep_); } + string + join(const set &set_, + const char sep_) + { + string rv; + set::iterator i; + + if(set_.empty()) + return string(); + + i = set_.begin(); + rv += *i; + ++i; + for(; i != set_.end(); ++i) + rv += sep_ + *i; + + return rv; + } + size_t longest_common_prefix_index(const vector &vec_) { diff --git a/src/str.hpp b/src/str.hpp index c9a6f268..34f19e8c 100644 --- a/src/str.hpp +++ b/src/str.hpp @@ -16,6 +16,7 @@ #pragma once +#include #include #include @@ -50,6 +51,10 @@ namespace str join(const std::vector &vec, const char sep); + std::string + join(const std::set &s, + const char sep); + size_t longest_common_prefix_index(const std::vector &vec); diff --git a/src/policy_invalid.cpp b/src/strvec.hpp similarity index 67% rename from src/policy_invalid.cpp rename to src/strvec.hpp index caedc24f..593263df 100644 --- a/src/policy_invalid.cpp +++ b/src/strvec.hpp @@ -1,5 +1,7 @@ /* - Copyright (c) 2016, Antonio SJ Musumeci + ISC License + + Copyright (c) 2020, Antonio SJ Musumeci Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above @@ -14,20 +16,9 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ -#include "errno.hpp" -#include "policy.hpp" +#pragma once #include #include -using std::string; -using std::vector; - -int -Policy::Func::invalid(const Category type_, - const Branches &branches_, - const char *fusepath_, - vector *paths_) -{ - return (errno=EINVAL,-1); -} +typedef std::vector StrVec; diff --git a/src/to_string.cpp b/src/to_string.cpp index 3019b682..0d5cddd7 100644 --- a/src/to_string.cpp +++ b/src/to_string.cpp @@ -16,10 +16,10 @@ OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. */ +#include #include #include -#include #include namespace str diff --git a/src/to_string.hpp b/src/to_string.hpp index 5e4f04e7..846147a4 100644 --- a/src/to_string.hpp +++ b/src/to_string.hpp @@ -18,9 +18,9 @@ #pragma once +#include #include -#include namespace str { diff --git a/src/tofrom_wrapper.hpp b/src/tofrom_wrapper.hpp index 9be0d3fe..ebcc605f 100644 --- a/src/tofrom_wrapper.hpp +++ b/src/tofrom_wrapper.hpp @@ -29,13 +29,13 @@ class ToFromWrapper : public ToFromString { public: int - from_string(const std::string &s_) + from_string(const std::string &s_) final { return str::from(s_,&_data); } std::string - to_string(void) const + to_string(void) const final { return str::to(_data); } @@ -70,6 +70,13 @@ public: return &_data; } + const + T* + operator->() const + { + return &_data; + } + public: bool operator==(const T &data_) const @@ -86,13 +93,13 @@ class ROToFromWrapper : public ToFromString { public: int - from_string(const std::string &s_) + from_string(const std::string &s_) final { return -EINVAL; } std::string - to_string(void) const + to_string(void) const final { return str::to(_data); } diff --git a/tests/acutest.h b/tests/acutest.h new file mode 100644 index 00000000..8b49afc8 --- /dev/null +++ b/tests/acutest.h @@ -0,0 +1,1794 @@ +/* + * Acutest -- Another C/C++ Unit Test facility + * + * + * Copyright 2013-2020 Martin Mitas + * Copyright 2019 Garrett D'Amore + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS + * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS + * IN THE SOFTWARE. + */ + +#ifndef ACUTEST_H +#define ACUTEST_H + + +/************************ + *** Public interface *** + ************************/ + +/* By default, "acutest.h" provides the main program entry point (function + * main()). However, if the test suite is composed of multiple source files + * which include "acutest.h", then this causes a problem of multiple main() + * definitions. To avoid this problem, #define macro TEST_NO_MAIN in all + * compilation units but one. + */ + +/* Macro to specify list of unit tests in the suite. + * The unit test implementation MUST provide list of unit tests it implements + * with this macro: + * + * TEST_LIST = { + * { "test1_name", test1_func_ptr }, + * { "test2_name", test2_func_ptr }, + * ... + * { NULL, NULL } // zeroed record marking the end of the list + * }; + * + * The list specifies names of each test (must be unique) and pointer to + * a function implementing it. The function does not take any arguments + * and has no return values, i.e. every test function has to be compatible + * with this prototype: + * + * void test_func(void); + * + * Note the list has to be ended with a zeroed record. + */ +#define TEST_LIST const struct test_ test_list_[] + + +/* Macros for testing whether an unit test succeeds or fails. These macros + * can be used arbitrarily in functions implementing the unit tests. + * + * If any condition fails throughout execution of a test, the test fails. + * + * TEST_CHECK takes only one argument (the condition), TEST_CHECK_ allows + * also to specify an error message to print out if the condition fails. + * (It expects printf-like format string and its parameters). The macros + * return non-zero (condition passes) or 0 (condition fails). + * + * That can be useful when more conditions should be checked only if some + * preceding condition passes, as illustrated in this code snippet: + * + * SomeStruct* ptr = allocate_some_struct(); + * if(TEST_CHECK(ptr != NULL)) { + * TEST_CHECK(ptr->member1 < 100); + * TEST_CHECK(ptr->member2 > 200); + * } + */ +#define TEST_CHECK_(cond,...) test_check_((cond), __FILE__, __LINE__, __VA_ARGS__) +#define TEST_CHECK(cond) test_check_((cond), __FILE__, __LINE__, "%s", #cond) + + +/* These macros are the same as TEST_CHECK_ and TEST_CHECK except that if the + * condition fails, the currently executed unit test is immediately aborted. + * + * That is done either by calling abort() if the unit test is executed as a + * child process; or via longjmp() if the unit test is executed within the + * main Acutest process. + * + * As a side effect of such abortion, your unit tests may cause memory leaks, + * unflushed file descriptors, and other phenomena caused by the abortion. + * + * Therefore you should not use these as a general replacement for TEST_CHECK. + * Use it with some caution, especially if your test causes some other side + * effects to the outside world (e.g. communicating with some server, inserting + * into a database etc.). + */ +#define TEST_ASSERT_(cond,...) \ + do { \ + if(!test_check_((cond), __FILE__, __LINE__, __VA_ARGS__)) \ + test_abort_(); \ + } while(0) +#define TEST_ASSERT(cond) \ + do { \ + if(!test_check_((cond), __FILE__, __LINE__, "%s", #cond)) \ + test_abort_(); \ + } while(0) + + +#ifdef __cplusplus +/* Macros to verify that the code (the 1st argument) throws exception of given + * type (the 2nd argument). (Note these macros are only available in C++.) + * + * TEST_EXCEPTION_ is like TEST_EXCEPTION but accepts custom printf-like + * message. + * + * For example: + * + * TEST_EXCEPTION(function_that_throw(), ExpectedExceptionType); + * + * If the function_that_throw() throws ExpectedExceptionType, the check passes. + * If the function throws anything incompatible with ExpectedExceptionType + * (or if it does not thrown an exception at all), the check fails. + */ +#define TEST_EXCEPTION(code, exctype) \ + do { \ + bool exc_ok_ = false; \ + const char *msg_ = NULL; \ + try { \ + code; \ + msg_ = "No exception thrown."; \ + } catch(exctype const&) { \ + exc_ok_= true; \ + } catch(...) { \ + msg_ = "Unexpected exception thrown."; \ + } \ + test_check_(exc_ok_, __FILE__, __LINE__, #code " throws " #exctype); \ + if(msg_ != NULL) \ + test_message_("%s", msg_); \ + } while(0) +#define TEST_EXCEPTION_(code, exctype, ...) \ + do { \ + bool exc_ok_ = false; \ + const char *msg_ = NULL; \ + try { \ + code; \ + msg_ = "No exception thrown."; \ + } catch(exctype const&) { \ + exc_ok_= true; \ + } catch(...) { \ + msg_ = "Unexpected exception thrown."; \ + } \ + test_check_(exc_ok_, __FILE__, __LINE__, __VA_ARGS__); \ + if(msg_ != NULL) \ + test_message_("%s", msg_); \ + } while(0) +#endif /* #ifdef __cplusplus */ + + +/* Sometimes it is useful to split execution of more complex unit tests to some + * smaller parts and associate those parts with some names. + * + * This is especially handy if the given unit test is implemented as a loop + * over some vector of multiple testing inputs. Using these macros allow to use + * sort of subtitle for each iteration of the loop (e.g. outputting the input + * itself or a name associated to it), so that if any TEST_CHECK condition + * fails in the loop, it can be easily seen which iteration triggers the + * failure, without the need to manually output the iteration-specific data in + * every single TEST_CHECK inside the loop body. + * + * TEST_CASE allows to specify only single string as the name of the case, + * TEST_CASE_ provides all the power of printf-like string formatting. + * + * Note that the test cases cannot be nested. Starting a new test case ends + * implicitly the previous one. To end the test case explicitly (e.g. to end + * the last test case after exiting the loop), you may use TEST_CASE(NULL). + */ +#define TEST_CASE_(...) test_case_(__VA_ARGS__) +#define TEST_CASE(name) test_case_("%s", name) + + +/* Maximal output per TEST_CASE call. Longer messages are cut. + * You may define another limit prior including "acutest.h" + */ +#ifndef TEST_CASE_MAXSIZE + #define TEST_CASE_MAXSIZE 64 +#endif + + +/* printf-like macro for outputting an extra information about a failure. + * + * Intended use is to output some computed output versus the expected value, + * e.g. like this: + * + * if(!TEST_CHECK(produced == expected)) { + * TEST_MSG("Expected: %d", expected); + * TEST_MSG("Produced: %d", produced); + * } + * + * Note the message is only written down if the most recent use of any checking + * macro (like e.g. TEST_CHECK or TEST_EXCEPTION) in the current test failed. + * This means the above is equivalent to just this: + * + * TEST_CHECK(produced == expected); + * TEST_MSG("Expected: %d", expected); + * TEST_MSG("Produced: %d", produced); + * + * The macro can deal with multi-line output fairly well. It also automatically + * adds a final new-line if there is none present. + */ +#define TEST_MSG(...) test_message_(__VA_ARGS__) + + +/* Maximal output per TEST_MSG call. Longer messages are cut. + * You may define another limit prior including "acutest.h" + */ +#ifndef TEST_MSG_MAXSIZE + #define TEST_MSG_MAXSIZE 1024 +#endif + + +/* Macro for dumping a block of memory. + * + * Its intended use is very similar to what TEST_MSG is for, but instead of + * generating any printf-like message, this is for dumping raw block of a + * memory in a hexadecimal form: + * + * TEST_CHECK(size_produced == size_expected && + * memcmp(addr_produced, addr_expected, size_produced) == 0); + * TEST_DUMP("Expected:", addr_expected, size_expected); + * TEST_DUMP("Produced:", addr_produced, size_produced); + */ +#define TEST_DUMP(title, addr, size) test_dump_(title, addr, size) + +/* Maximal output per TEST_DUMP call (in bytes to dump). Longer blocks are cut. + * You may define another limit prior including "acutest.h" + */ +#ifndef TEST_DUMP_MAXSIZE + #define TEST_DUMP_MAXSIZE 1024 +#endif + + +/* Common test initialiation/clean-up + * + * In some test suites, it may be needed to perform some sort of the same + * initialization and/or clean-up in all the tests. + * + * Such test suites may use macros TEST_INIT and/or TEST_FINI prior including + * this header. The expansion of the macro is then used as a body of helper + * function called just before executing every single (TEST_INIT) or just after + * it ends (TEST_FINI). + * + * Examples of various ways how to use the macro TEST_INIT: + * + * #define TEST_INIT my_init_func(); + * #define TEST_INIT my_init_func() // Works even without the semicolon + * #define TEST_INIT setlocale(LC_ALL, NULL); + * #define TEST_INIT { setlocale(LC_ALL, NULL); my_init_func(); } + * + * TEST_FINI is to be used in the same way. + */ + + +/********************** + *** Implementation *** + **********************/ + +/* The unit test files should not rely on anything below. */ + +#include +#include +#include +#include +#include +#include + +#if defined(unix) || defined(__unix__) || defined(__unix) || defined(__APPLE__) + #define ACUTEST_UNIX_ 1 + #include + #include + #include + #include + #include + #include + #include + + #if defined CLOCK_PROCESS_CPUTIME_ID && defined CLOCK_MONOTONIC + #define ACUTEST_HAS_POSIX_TIMER_ 1 + #endif +#endif + +#if defined(_gnu_linux_) || defined(__linux__) + #define ACUTEST_LINUX_ 1 + #include + #include +#endif + +#if defined(_WIN32) || defined(__WIN32__) || defined(__WINDOWS__) + #define ACUTEST_WIN_ 1 + #include + #include +#endif + +#ifdef __cplusplus + #include +#endif + +/* Load valgrind.h, if available. This allows to detect valgrind's presence via RUNNING_ON_VALGRIND. */ +#ifdef __has_include + #if __has_include() + #include + #endif +#endif + +/* Enable the use of the non-standard keyword __attribute__ to silence warnings under some compilers */ +#if defined(__GNUC__) || defined(__clang__) + #define TEST_ATTRIBUTE_(attr) __attribute__((attr)) +#else + #define TEST_ATTRIBUTE_(attr) +#endif + +/* Note our global private identifiers end with '_' to mitigate risk of clash + * with the unit tests implementation. */ + +#ifdef __cplusplus + extern "C" { +#endif + +#ifdef _MSC_VER + /* In the multi-platform code like ours, we cannot use the non-standard + * "safe" functions from Microsoft C lib like e.g. sprintf_s() instead of + * standard sprintf(). Hence, lets disable the warning C4996. */ + #pragma warning(push) + #pragma warning(disable: 4996) +#endif + + +struct test_ { + const char* name; + void (*func)(void); +}; + +struct test_detail_ { + unsigned char flags; + double duration; +}; + +enum { + TEST_FLAG_RUN_ = 1 << 0, + TEST_FLAG_SUCCESS_ = 1 << 1, + TEST_FLAG_FAILURE_ = 1 << 2, +}; + +extern const struct test_ test_list_[]; + +int test_check_(int cond, const char* file, int line, const char* fmt, ...); +void test_case_(const char* fmt, ...); +void test_message_(const char* fmt, ...); +void test_dump_(const char* title, const void* addr, size_t size); +void test_abort_(void) TEST_ATTRIBUTE_(noreturn); + + +#ifndef TEST_NO_MAIN + +static char* test_argv0_ = NULL; +static size_t test_list_size_ = 0; +static struct test_detail_ *test_details_ = NULL; +static size_t test_count_ = 0; +static int test_no_exec_ = -1; +static int test_no_summary_ = 0; +static int test_tap_ = 0; +static int test_skip_mode_ = 0; +static int test_worker_ = 0; +static int test_worker_index_ = 0; +static int test_cond_failed_ = 0; +static int test_was_aborted_ = 0; +static FILE *test_xml_output_ = NULL; + +static int test_stat_failed_units_ = 0; +static int test_stat_run_units_ = 0; + +static const struct test_* test_current_unit_ = NULL; +static int test_current_index_ = 0; +static char test_case_name_[TEST_CASE_MAXSIZE] = ""; +static int test_current_already_logged_ = 0; +static int test_case_current_already_logged_ = 0; +static int test_verbose_level_ = 2; +static int test_current_failures_ = 0; +static int test_colorize_ = 0; +static int test_timer_ = 0; + +static int test_abort_has_jmp_buf_ = 0; +static jmp_buf test_abort_jmp_buf_; + + +static void +test_cleanup_(void) +{ + free((void*) test_details_); +} + +static void +test_exit_(int exit_code) +{ + test_cleanup_(); + exit(exit_code); +} + +#if defined ACUTEST_WIN_ + typedef LARGE_INTEGER test_timer_type_; + static LARGE_INTEGER test_timer_freq_; + static test_timer_type_ test_timer_start_; + static test_timer_type_ test_timer_end_; + + static void + test_timer_init_(void) + { + QueryPerformanceFrequency(&test_timer_freq_); + } + + static void + test_timer_get_time_(LARGE_INTEGER* ts) + { + QueryPerformanceCounter(ts); + } + + static double + test_timer_diff_(LARGE_INTEGER start, LARGE_INTEGER end) + { + double duration = (double)(end.QuadPart - start.QuadPart); + duration /= (double)test_timer_freq_.QuadPart; + return duration; + } + + static void + test_timer_print_diff_(void) + { + printf("%.6lf secs", test_timer_diff_(test_timer_start_, test_timer_end_)); + } +#elif defined ACUTEST_HAS_POSIX_TIMER_ + static clockid_t test_timer_id_; + typedef struct timespec test_timer_type_; + static test_timer_type_ test_timer_start_; + static test_timer_type_ test_timer_end_; + + static void + test_timer_init_(void) + { + if(test_timer_ == 1) + test_timer_id_ = CLOCK_MONOTONIC; + else if(test_timer_ == 2) + test_timer_id_ = CLOCK_PROCESS_CPUTIME_ID; + } + + static void + test_timer_get_time_(struct timespec* ts) + { + clock_gettime(test_timer_id_, ts); + } + + static double + test_timer_diff_(struct timespec start, struct timespec end) + { + double endns; + double startns; + + endns = end.tv_sec; + endns *= 1e9; + endns += end.tv_nsec; + + startns = start.tv_sec; + startns *= 1e9; + startns += start.tv_nsec; + + return ((endns - startns)/ 1e9); + } + + static void + test_timer_print_diff_(void) + { + printf("%.6lf secs", + test_timer_diff_(test_timer_start_, test_timer_end_)); + } +#else + typedef int test_timer_type_; + static test_timer_type_ test_timer_start_; + static test_timer_type_ test_timer_end_; + + void + test_timer_init_(void) + {} + + static void + test_timer_get_time_(int* ts) + { + (void) ts; + } + + static double + test_timer_diff_(int start, int end) + { + (void) start; + (void) end; + return 0.0; + } + + static void + test_timer_print_diff_(void) + {} +#endif + +#define TEST_COLOR_DEFAULT_ 0 +#define TEST_COLOR_GREEN_ 1 +#define TEST_COLOR_RED_ 2 +#define TEST_COLOR_DEFAULT_INTENSIVE_ 3 +#define TEST_COLOR_GREEN_INTENSIVE_ 4 +#define TEST_COLOR_RED_INTENSIVE_ 5 + +static int TEST_ATTRIBUTE_(format (printf, 2, 3)) +test_print_in_color_(int color, const char* fmt, ...) +{ + va_list args; + char buffer[256]; + int n; + + va_start(args, fmt); + vsnprintf(buffer, sizeof(buffer), fmt, args); + va_end(args); + buffer[sizeof(buffer)-1] = '\0'; + + if(!test_colorize_) { + return printf("%s", buffer); + } + +#if defined ACUTEST_UNIX_ + { + const char* col_str; + switch(color) { + case TEST_COLOR_GREEN_: col_str = "\033[0;32m"; break; + case TEST_COLOR_RED_: col_str = "\033[0;31m"; break; + case TEST_COLOR_GREEN_INTENSIVE_: col_str = "\033[1;32m"; break; + case TEST_COLOR_RED_INTENSIVE_: col_str = "\033[1;31m"; break; + case TEST_COLOR_DEFAULT_INTENSIVE_: col_str = "\033[1m"; break; + default: col_str = "\033[0m"; break; + } + printf("%s", col_str); + n = printf("%s", buffer); + printf("\033[0m"); + return n; + } +#elif defined ACUTEST_WIN_ + { + HANDLE h; + CONSOLE_SCREEN_BUFFER_INFO info; + WORD attr; + + h = GetStdHandle(STD_OUTPUT_HANDLE); + GetConsoleScreenBufferInfo(h, &info); + + switch(color) { + case TEST_COLOR_GREEN_: attr = FOREGROUND_GREEN; break; + case TEST_COLOR_RED_: attr = FOREGROUND_RED; break; + case TEST_COLOR_GREEN_INTENSIVE_: attr = FOREGROUND_GREEN | FOREGROUND_INTENSITY; break; + case TEST_COLOR_RED_INTENSIVE_: attr = FOREGROUND_RED | FOREGROUND_INTENSITY; break; + case TEST_COLOR_DEFAULT_INTENSIVE_: attr = FOREGROUND_BLUE | FOREGROUND_GREEN | FOREGROUND_RED | FOREGROUND_INTENSITY; break; + default: attr = 0; break; + } + if(attr != 0) + SetConsoleTextAttribute(h, attr); + n = printf("%s", buffer); + SetConsoleTextAttribute(h, info.wAttributes); + return n; + } +#else + n = printf("%s", buffer); + return n; +#endif +} + +static void +test_begin_test_line_(const struct test_* test) +{ + if(!test_tap_) { + if(test_verbose_level_ >= 3) { + test_print_in_color_(TEST_COLOR_DEFAULT_INTENSIVE_, "Test %s:\n", test->name); + test_current_already_logged_++; + } else if(test_verbose_level_ >= 1) { + int n; + char spaces[48]; + + n = test_print_in_color_(TEST_COLOR_DEFAULT_INTENSIVE_, "Test %s... ", test->name); + memset(spaces, ' ', sizeof(spaces)); + if(n < (int) sizeof(spaces)) + printf("%.*s", (int) sizeof(spaces) - n, spaces); + } else { + test_current_already_logged_ = 1; + } + } +} + +static void +test_finish_test_line_(int result) +{ + if(test_tap_) { + const char* str = (result == 0) ? "ok" : "not ok"; + + printf("%s %d - %s\n", str, test_current_index_ + 1, test_current_unit_->name); + + if(result == 0 && test_timer_) { + printf("# Duration: "); + test_timer_print_diff_(); + printf("\n"); + } + } else { + int color = (result == 0) ? TEST_COLOR_GREEN_INTENSIVE_ : TEST_COLOR_RED_INTENSIVE_; + const char* str = (result == 0) ? "OK" : "FAILED"; + printf("[ "); + test_print_in_color_(color, "%s", str); + printf(" ]"); + + if(result == 0 && test_timer_) { + printf(" "); + test_timer_print_diff_(); + } + + printf("\n"); + } +} + +static void +test_line_indent_(int level) +{ + static const char spaces[] = " "; + int n = level * 2; + + if(test_tap_ && n > 0) { + n--; + printf("#"); + } + + while(n > 16) { + printf("%s", spaces); + n -= 16; + } + printf("%.*s", n, spaces); +} + +int TEST_ATTRIBUTE_(format (printf, 4, 5)) +test_check_(int cond, const char* file, int line, const char* fmt, ...) +{ + const char *result_str; + int result_color; + int verbose_level; + + if(cond) { + result_str = "ok"; + result_color = TEST_COLOR_GREEN_; + verbose_level = 3; + } else { + if(!test_current_already_logged_ && test_current_unit_ != NULL) + test_finish_test_line_(-1); + + result_str = "failed"; + result_color = TEST_COLOR_RED_; + verbose_level = 2; + test_current_failures_++; + test_current_already_logged_++; + } + + if(test_verbose_level_ >= verbose_level) { + va_list args; + + if(!test_case_current_already_logged_ && test_case_name_[0]) { + test_line_indent_(1); + test_print_in_color_(TEST_COLOR_DEFAULT_INTENSIVE_, "Case %s:\n", test_case_name_); + test_current_already_logged_++; + test_case_current_already_logged_++; + } + + test_line_indent_(test_case_name_[0] ? 2 : 1); + if(file != NULL) { +#ifdef ACUTEST_WIN_ + const char* lastsep1 = strrchr(file, '\\'); + const char* lastsep2 = strrchr(file, '/'); + if(lastsep1 == NULL) + lastsep1 = file-1; + if(lastsep2 == NULL) + lastsep2 = file-1; + file = (lastsep1 > lastsep2 ? lastsep1 : lastsep2) + 1; +#else + const char* lastsep = strrchr(file, '/'); + if(lastsep != NULL) + file = lastsep+1; +#endif + printf("%s:%d: Check ", file, line); + } + + va_start(args, fmt); + vprintf(fmt, args); + va_end(args); + + printf("... "); + test_print_in_color_(result_color, "%s", result_str); + printf("\n"); + test_current_already_logged_++; + } + + test_cond_failed_ = (cond == 0); + return !test_cond_failed_; +} + +void TEST_ATTRIBUTE_(format (printf, 1, 2)) +test_case_(const char* fmt, ...) +{ + va_list args; + + if(test_verbose_level_ < 2) + return; + + if(test_case_name_[0]) { + test_case_current_already_logged_ = 0; + test_case_name_[0] = '\0'; + } + + if(fmt == NULL) + return; + + va_start(args, fmt); + vsnprintf(test_case_name_, sizeof(test_case_name_) - 1, fmt, args); + va_end(args); + test_case_name_[sizeof(test_case_name_) - 1] = '\0'; + + if(test_verbose_level_ >= 3) { + test_line_indent_(1); + test_print_in_color_(TEST_COLOR_DEFAULT_INTENSIVE_, "Case %s:\n", test_case_name_); + test_current_already_logged_++; + test_case_current_already_logged_++; + } +} + +void TEST_ATTRIBUTE_(format (printf, 1, 2)) +test_message_(const char* fmt, ...) +{ + char buffer[TEST_MSG_MAXSIZE]; + char* line_beg; + char* line_end; + va_list args; + + if(test_verbose_level_ < 2) + return; + + /* We allow extra message only when something is already wrong in the + * current test. */ + if(test_current_unit_ == NULL || !test_cond_failed_) + return; + + va_start(args, fmt); + vsnprintf(buffer, TEST_MSG_MAXSIZE, fmt, args); + va_end(args); + buffer[TEST_MSG_MAXSIZE-1] = '\0'; + + line_beg = buffer; + while(1) { + line_end = strchr(line_beg, '\n'); + if(line_end == NULL) + break; + test_line_indent_(test_case_name_[0] ? 3 : 2); + printf("%.*s\n", (int)(line_end - line_beg), line_beg); + line_beg = line_end + 1; + } + if(line_beg[0] != '\0') { + test_line_indent_(test_case_name_[0] ? 3 : 2); + printf("%s\n", line_beg); + } +} + +void +test_dump_(const char* title, const void* addr, size_t size) +{ + static const size_t BYTES_PER_LINE = 16; + size_t line_beg; + size_t truncate = 0; + + if(test_verbose_level_ < 2) + return; + + /* We allow extra message only when something is already wrong in the + * current test. */ + if(test_current_unit_ == NULL || !test_cond_failed_) + return; + + if(size > TEST_DUMP_MAXSIZE) { + truncate = size - TEST_DUMP_MAXSIZE; + size = TEST_DUMP_MAXSIZE; + } + + test_line_indent_(test_case_name_[0] ? 3 : 2); + printf((title[strlen(title)-1] == ':') ? "%s\n" : "%s:\n", title); + + for(line_beg = 0; line_beg < size; line_beg += BYTES_PER_LINE) { + size_t line_end = line_beg + BYTES_PER_LINE; + size_t off; + + test_line_indent_(test_case_name_[0] ? 4 : 3); + printf("%08lx: ", (unsigned long)line_beg); + for(off = line_beg; off < line_end; off++) { + if(off < size) + printf(" %02x", ((const unsigned char*)addr)[off]); + else + printf(" "); + } + + printf(" "); + for(off = line_beg; off < line_end; off++) { + unsigned char byte = ((const unsigned char*)addr)[off]; + if(off < size) + printf("%c", (iscntrl(byte) ? '.' : byte)); + else + break; + } + + printf("\n"); + } + + if(truncate > 0) { + test_line_indent_(test_case_name_[0] ? 4 : 3); + printf(" ... (and more %u bytes)\n", (unsigned) truncate); + } +} + +/* This is called just before each test */ +static void +test_init_(const char *test_name) +{ +#ifdef TEST_INIT + TEST_INIT + ; /* Allow for a single unterminated function call */ +#endif + + /* Suppress any warnings about unused variable. */ + (void) test_name; +} + +/* This is called after each test */ +static void +test_fini_(const char *test_name) +{ +#ifdef TEST_FINI + TEST_FINI + ; /* Allow for a single unterminated function call */ +#endif + + /* Suppress any warnings about unused variable. */ + (void) test_name; +} + +void +test_abort_(void) +{ + if(test_abort_has_jmp_buf_) { + longjmp(test_abort_jmp_buf_, 1); + } else { + if(test_current_unit_ != NULL) + test_fini_(test_current_unit_->name); + abort(); + } +} + +static void +test_list_names_(void) +{ + const struct test_* test; + + printf("Unit tests:\n"); + for(test = &test_list_[0]; test->func != NULL; test++) + printf(" %s\n", test->name); +} + +static void +test_remember_(int i) +{ + if(test_details_[i].flags & TEST_FLAG_RUN_) + return; + + test_details_[i].flags |= TEST_FLAG_RUN_; + test_count_++; +} + +static void +test_set_success_(int i, int success) +{ + test_details_[i].flags |= success ? TEST_FLAG_SUCCESS_ : TEST_FLAG_FAILURE_; +} + +static void +test_set_duration_(int i, double duration) +{ + test_details_[i].duration = duration; +} + +static int +test_name_contains_word_(const char* name, const char* pattern) +{ + static const char word_delim[] = " \t-_/.,:;"; + const char* substr; + size_t pattern_len; + + pattern_len = strlen(pattern); + + substr = strstr(name, pattern); + while(substr != NULL) { + int starts_on_word_boundary = (substr == name || strchr(word_delim, substr[-1]) != NULL); + int ends_on_word_boundary = (substr[pattern_len] == '\0' || strchr(word_delim, substr[pattern_len]) != NULL); + + if(starts_on_word_boundary && ends_on_word_boundary) + return 1; + + substr = strstr(substr+1, pattern); + } + + return 0; +} + +static int +test_lookup_(const char* pattern) +{ + int i; + int n = 0; + + /* Try exact match. */ + for(i = 0; i < (int) test_list_size_; i++) { + if(strcmp(test_list_[i].name, pattern) == 0) { + test_remember_(i); + n++; + break; + } + } + if(n > 0) + return n; + + /* Try word match. */ + for(i = 0; i < (int) test_list_size_; i++) { + if(test_name_contains_word_(test_list_[i].name, pattern)) { + test_remember_(i); + n++; + } + } + if(n > 0) + return n; + + /* Try relaxed match. */ + for(i = 0; i < (int) test_list_size_; i++) { + if(strstr(test_list_[i].name, pattern) != NULL) { + test_remember_(i); + n++; + } + } + + return n; +} + + +/* Called if anything goes bad in Acutest, or if the unit test ends in other + * way then by normal returning from its function (e.g. exception or some + * abnormal child process termination). */ +static void TEST_ATTRIBUTE_(format (printf, 1, 2)) +test_error_(const char* fmt, ...) +{ + if(test_verbose_level_ == 0) + return; + + if(test_verbose_level_ >= 2) { + va_list args; + + test_line_indent_(1); + if(test_verbose_level_ >= 3) + test_print_in_color_(TEST_COLOR_RED_INTENSIVE_, "ERROR: "); + va_start(args, fmt); + vprintf(fmt, args); + va_end(args); + printf("\n"); + } + + if(test_verbose_level_ >= 3) { + printf("\n"); + } +} + +/* Call directly the given test unit function. */ +static int +test_do_run_(const struct test_* test, int index) +{ + int status = -1; + + test_was_aborted_ = 0; + test_current_unit_ = test; + test_current_index_ = index; + test_current_failures_ = 0; + test_current_already_logged_ = 0; + test_cond_failed_ = 0; + +#ifdef __cplusplus + try { +#endif + test_init_(test->name); + test_begin_test_line_(test); + + /* This is good to do in case the test unit crashes. */ + fflush(stdout); + fflush(stderr); + + if(!test_worker_) { + test_abort_has_jmp_buf_ = 1; + if(setjmp(test_abort_jmp_buf_) != 0) { + test_was_aborted_ = 1; + goto aborted; + } + } + + test_timer_get_time_(&test_timer_start_); + test->func(); +aborted: + test_abort_has_jmp_buf_ = 0; + test_timer_get_time_(&test_timer_end_); + + if(test_verbose_level_ >= 3) { + test_line_indent_(1); + if(test_current_failures_ == 0) { + test_print_in_color_(TEST_COLOR_GREEN_INTENSIVE_, "SUCCESS: "); + printf("All conditions have passed.\n"); + + if(test_timer_) { + test_line_indent_(1); + printf("Duration: "); + test_timer_print_diff_(); + printf("\n"); + } + } else { + test_print_in_color_(TEST_COLOR_RED_INTENSIVE_, "FAILED: "); + if(!test_was_aborted_) { + printf("%d condition%s %s failed.\n", + test_current_failures_, + (test_current_failures_ == 1) ? "" : "s", + (test_current_failures_ == 1) ? "has" : "have"); + } else { + printf("Aborted.\n"); + } + } + printf("\n"); + } else if(test_verbose_level_ >= 1 && test_current_failures_ == 0) { + test_finish_test_line_(0); + } + + status = (test_current_failures_ == 0) ? 0 : -1; + +#ifdef __cplusplus + } catch(std::exception& e) { + const char* what = e.what(); + test_check_(0, NULL, 0, "Threw std::exception"); + if(what != NULL) + test_message_("std::exception::what(): %s", what); + + if(test_verbose_level_ >= 3) { + test_line_indent_(1); + test_print_in_color_(TEST_COLOR_RED_INTENSIVE_, "FAILED: "); + printf("C++ exception.\n\n"); + } + } catch(...) { + test_check_(0, NULL, 0, "Threw an exception"); + + if(test_verbose_level_ >= 3) { + test_line_indent_(1); + test_print_in_color_(TEST_COLOR_RED_INTENSIVE_, "FAILED: "); + printf("C++ exception.\n\n"); + } + } +#endif + + test_fini_(test->name); + test_case_(NULL); + test_current_unit_ = NULL; + + return status; +} + +/* Trigger the unit test. If possible (and not suppressed) it starts a child + * process who calls test_do_run_(), otherwise it calls test_do_run_() + * directly. */ +static void +test_run_(const struct test_* test, int index, int master_index) +{ + int failed = 1; + test_timer_type_ start, end; + + test_current_unit_ = test; + test_current_already_logged_ = 0; + test_timer_get_time_(&start); + + if(!test_no_exec_) { + +#if defined(ACUTEST_UNIX_) + + pid_t pid; + int exit_code; + + /* Make sure the child starts with empty I/O buffers. */ + fflush(stdout); + fflush(stderr); + + pid = fork(); + if(pid == (pid_t)-1) { + test_error_("Cannot fork. %s [%d]", strerror(errno), errno); + failed = 1; + } else if(pid == 0) { + /* Child: Do the test. */ + test_worker_ = 1; + failed = (test_do_run_(test, index) != 0); + test_exit_(failed ? 1 : 0); + } else { + /* Parent: Wait until child terminates and analyze its exit code. */ + waitpid(pid, &exit_code, 0); + if(WIFEXITED(exit_code)) { + switch(WEXITSTATUS(exit_code)) { + case 0: failed = 0; break; /* test has passed. */ + case 1: /* noop */ break; /* "normal" failure. */ + default: test_error_("Unexpected exit code [%d]", WEXITSTATUS(exit_code)); + } + } else if(WIFSIGNALED(exit_code)) { + char tmp[32]; + const char* signame; + switch(WTERMSIG(exit_code)) { + case SIGINT: signame = "SIGINT"; break; + case SIGHUP: signame = "SIGHUP"; break; + case SIGQUIT: signame = "SIGQUIT"; break; + case SIGABRT: signame = "SIGABRT"; break; + case SIGKILL: signame = "SIGKILL"; break; + case SIGSEGV: signame = "SIGSEGV"; break; + case SIGILL: signame = "SIGILL"; break; + case SIGTERM: signame = "SIGTERM"; break; + default: sprintf(tmp, "signal %d", WTERMSIG(exit_code)); signame = tmp; break; + } + test_error_("Test interrupted by %s.", signame); + } else { + test_error_("Test ended in an unexpected way [%d].", exit_code); + } + } + +#elif defined(ACUTEST_WIN_) + + char buffer[512] = {0}; + STARTUPINFOA startupInfo; + PROCESS_INFORMATION processInfo; + DWORD exitCode; + + /* Windows has no fork(). So we propagate all info into the child + * through a command line arguments. */ + _snprintf(buffer, sizeof(buffer)-1, + "%s --worker=%d %s --no-exec --no-summary %s --verbose=%d --color=%s -- \"%s\"", + test_argv0_, index, test_timer_ ? "--time" : "", + test_tap_ ? "--tap" : "", test_verbose_level_, + test_colorize_ ? "always" : "never", + test->name); + memset(&startupInfo, 0, sizeof(startupInfo)); + startupInfo.cb = sizeof(STARTUPINFO); + if(CreateProcessA(NULL, buffer, NULL, NULL, FALSE, 0, NULL, NULL, &startupInfo, &processInfo)) { + WaitForSingleObject(processInfo.hProcess, INFINITE); + GetExitCodeProcess(processInfo.hProcess, &exitCode); + CloseHandle(processInfo.hThread); + CloseHandle(processInfo.hProcess); + failed = (exitCode != 0); + if(exitCode > 1) { + switch(exitCode) { + case 3: test_error_("Aborted."); break; + case 0xC0000005: test_error_("Access violation."); break; + default: test_error_("Test ended in an unexpected way [%lu].", exitCode); break; + } + } + } else { + test_error_("Cannot create unit test subprocess [%ld].", GetLastError()); + failed = 1; + } + +#else + + /* A platform where we don't know how to run child process. */ + failed = (test_do_run_(test, index) != 0); + +#endif + + } else { + /* Child processes suppressed through --no-exec. */ + failed = (test_do_run_(test, index) != 0); + } + test_timer_get_time_(&end); + + test_current_unit_ = NULL; + + test_stat_run_units_++; + if(failed) + test_stat_failed_units_++; + + test_set_success_(master_index, !failed); + test_set_duration_(master_index, test_timer_diff_(start, end)); +} + +#if defined(ACUTEST_WIN_) +/* Callback for SEH events. */ +static LONG CALLBACK +test_seh_exception_filter_(EXCEPTION_POINTERS *ptrs) +{ + test_check_(0, NULL, 0, "Unhandled SEH exception"); + test_message_("Exception code: 0x%08lx", ptrs->ExceptionRecord->ExceptionCode); + test_message_("Exception address: 0x%p", ptrs->ExceptionRecord->ExceptionAddress); + + fflush(stdout); + fflush(stderr); + + return EXCEPTION_EXECUTE_HANDLER; +} +#endif + + +#define TEST_CMDLINE_OPTFLAG_OPTIONALARG_ 0x0001 +#define TEST_CMDLINE_OPTFLAG_REQUIREDARG_ 0x0002 + +#define TEST_CMDLINE_OPTID_NONE_ 0 +#define TEST_CMDLINE_OPTID_UNKNOWN_ (-0x7fffffff + 0) +#define TEST_CMDLINE_OPTID_MISSINGARG_ (-0x7fffffff + 1) +#define TEST_CMDLINE_OPTID_BOGUSARG_ (-0x7fffffff + 2) + +typedef struct TEST_CMDLINE_OPTION_ { + char shortname; + const char* longname; + int id; + unsigned flags; +} TEST_CMDLINE_OPTION_; + +static int +test_cmdline_handle_short_opt_group_(const TEST_CMDLINE_OPTION_* options, + const char* arggroup, + int (*callback)(int /*optval*/, const char* /*arg*/)) +{ + const TEST_CMDLINE_OPTION_* opt; + int i; + int ret = 0; + + for(i = 0; arggroup[i] != '\0'; i++) { + for(opt = options; opt->id != 0; opt++) { + if(arggroup[i] == opt->shortname) + break; + } + + if(opt->id != 0 && !(opt->flags & TEST_CMDLINE_OPTFLAG_REQUIREDARG_)) { + ret = callback(opt->id, NULL); + } else { + /* Unknown option. */ + char badoptname[3]; + badoptname[0] = '-'; + badoptname[1] = arggroup[i]; + badoptname[2] = '\0'; + ret = callback((opt->id != 0 ? TEST_CMDLINE_OPTID_MISSINGARG_ : TEST_CMDLINE_OPTID_UNKNOWN_), + badoptname); + } + + if(ret != 0) + break; + } + + return ret; +} + +#define TEST_CMDLINE_AUXBUF_SIZE_ 32 + +static int +test_cmdline_read_(const TEST_CMDLINE_OPTION_* options, int argc, char** argv, + int (*callback)(int /*optval*/, const char* /*arg*/)) +{ + + const TEST_CMDLINE_OPTION_* opt; + char auxbuf[TEST_CMDLINE_AUXBUF_SIZE_+1]; + int after_doubledash = 0; + int i = 1; + int ret = 0; + + auxbuf[TEST_CMDLINE_AUXBUF_SIZE_] = '\0'; + + while(i < argc) { + if(after_doubledash || strcmp(argv[i], "-") == 0) { + /* Non-option argument. */ + ret = callback(TEST_CMDLINE_OPTID_NONE_, argv[i]); + } else if(strcmp(argv[i], "--") == 0) { + /* End of options. All the remaining members are non-option arguments. */ + after_doubledash = 1; + } else if(argv[i][0] != '-') { + /* Non-option argument. */ + ret = callback(TEST_CMDLINE_OPTID_NONE_, argv[i]); + } else { + for(opt = options; opt->id != 0; opt++) { + if(opt->longname != NULL && strncmp(argv[i], "--", 2) == 0) { + size_t len = strlen(opt->longname); + if(strncmp(argv[i]+2, opt->longname, len) == 0) { + /* Regular long option. */ + if(argv[i][2+len] == '\0') { + /* with no argument provided. */ + if(!(opt->flags & TEST_CMDLINE_OPTFLAG_REQUIREDARG_)) + ret = callback(opt->id, NULL); + else + ret = callback(TEST_CMDLINE_OPTID_MISSINGARG_, argv[i]); + break; + } else if(argv[i][2+len] == '=') { + /* with an argument provided. */ + if(opt->flags & (TEST_CMDLINE_OPTFLAG_OPTIONALARG_ | TEST_CMDLINE_OPTFLAG_REQUIREDARG_)) { + ret = callback(opt->id, argv[i]+2+len+1); + } else { + sprintf(auxbuf, "--%s", opt->longname); + ret = callback(TEST_CMDLINE_OPTID_BOGUSARG_, auxbuf); + } + break; + } else { + continue; + } + } + } else if(opt->shortname != '\0' && argv[i][0] == '-') { + if(argv[i][1] == opt->shortname) { + /* Regular short option. */ + if(opt->flags & TEST_CMDLINE_OPTFLAG_REQUIREDARG_) { + if(argv[i][2] != '\0') + ret = callback(opt->id, argv[i]+2); + else if(i+1 < argc) + ret = callback(opt->id, argv[++i]); + else + ret = callback(TEST_CMDLINE_OPTID_MISSINGARG_, argv[i]); + break; + } else { + ret = callback(opt->id, NULL); + + /* There might be more (argument-less) short options + * grouped together. */ + if(ret == 0 && argv[i][2] != '\0') + ret = test_cmdline_handle_short_opt_group_(options, argv[i]+2, callback); + break; + } + } + } + } + + if(opt->id == 0) { /* still not handled? */ + if(argv[i][0] != '-') { + /* Non-option argument. */ + ret = callback(TEST_CMDLINE_OPTID_NONE_, argv[i]); + } else { + /* Unknown option. */ + char* badoptname = argv[i]; + + if(strncmp(badoptname, "--", 2) == 0) { + /* Strip any argument from the long option. */ + char* assignment = strchr(badoptname, '='); + if(assignment != NULL) { + size_t len = assignment - badoptname; + if(len > TEST_CMDLINE_AUXBUF_SIZE_) + len = TEST_CMDLINE_AUXBUF_SIZE_; + strncpy(auxbuf, badoptname, len); + auxbuf[len] = '\0'; + badoptname = auxbuf; + } + } + + ret = callback(TEST_CMDLINE_OPTID_UNKNOWN_, badoptname); + } + } + } + + if(ret != 0) + return ret; + i++; + } + + return ret; +} + +static void +test_help_(void) +{ + printf("Usage: %s [options] [test...]\n", test_argv0_); + printf("\n"); + printf("Run the specified unit tests; or if the option '--skip' is used, run all\n"); + printf("tests in the suite but those listed. By default, if no tests are specified\n"); + printf("on the command line, all unit tests in the suite are run.\n"); + printf("\n"); + printf("Options:\n"); + printf(" -s, --skip Execute all unit tests but the listed ones\n"); + printf(" --exec[=WHEN] If supported, execute unit tests as child processes\n"); + printf(" (WHEN is one of 'auto', 'always', 'never')\n"); + printf(" -E, --no-exec Same as --exec=never\n"); +#if defined ACUTEST_WIN_ + printf(" -t, --time Measure test duration\n"); +#elif defined ACUTEST_HAS_POSIX_TIMER_ + printf(" -t, --time Measure test duration (real time)\n"); + printf(" --time=TIMER Measure test duration, using given timer\n"); + printf(" (TIMER is one of 'real', 'cpu')\n"); +#endif + printf(" --no-summary Suppress printing of test results summary\n"); + printf(" --tap Produce TAP-compliant output\n"); + printf(" (See https://testanything.org/)\n"); + printf(" -x, --xml-output=FILE Enable XUnit output to the given file\n"); + printf(" -l, --list List unit tests in the suite and exit\n"); + printf(" -v, --verbose Make output more verbose\n"); + printf(" --verbose=LEVEL Set verbose level to LEVEL:\n"); + printf(" 0 ... Be silent\n"); + printf(" 1 ... Output one line per test (and summary)\n"); + printf(" 2 ... As 1 and failed conditions (this is default)\n"); + printf(" 3 ... As 1 and all conditions (and extended summary)\n"); + printf(" -q, --quiet Same as --verbose=0\n"); + printf(" --color[=WHEN] Enable colorized output\n"); + printf(" (WHEN is one of 'auto', 'always', 'never')\n"); + printf(" --no-color Same as --color=never\n"); + printf(" -h, --help Display this help and exit\n"); + + if(test_list_size_ < 16) { + printf("\n"); + test_list_names_(); + } +} + +static const TEST_CMDLINE_OPTION_ test_cmdline_options_[] = { + { 's', "skip", 's', 0 }, + { 0, "exec", 'e', TEST_CMDLINE_OPTFLAG_OPTIONALARG_ }, + { 'E', "no-exec", 'E', 0 }, +#if defined ACUTEST_WIN_ + { 't', "time", 't', 0 }, + { 0, "timer", 't', 0 }, /* kept for compatibility */ +#elif defined ACUTEST_HAS_POSIX_TIMER_ + { 't', "time", 't', TEST_CMDLINE_OPTFLAG_OPTIONALARG_ }, + { 0, "timer", 't', TEST_CMDLINE_OPTFLAG_OPTIONALARG_ }, /* kept for compatibility */ +#endif + { 0, "no-summary", 'S', 0 }, + { 0, "tap", 'T', 0 }, + { 'l', "list", 'l', 0 }, + { 'v', "verbose", 'v', TEST_CMDLINE_OPTFLAG_OPTIONALARG_ }, + { 'q', "quiet", 'q', 0 }, + { 0, "color", 'c', TEST_CMDLINE_OPTFLAG_OPTIONALARG_ }, + { 0, "no-color", 'C', 0 }, + { 'h', "help", 'h', 0 }, + { 0, "worker", 'w', TEST_CMDLINE_OPTFLAG_REQUIREDARG_ }, /* internal */ + { 'x', "xml-output", 'x', TEST_CMDLINE_OPTFLAG_REQUIREDARG_ }, + { 0, NULL, 0, 0 } +}; + +static int +test_cmdline_callback_(int id, const char* arg) +{ + switch(id) { + case 's': + test_skip_mode_ = 1; + break; + + case 'e': + if(arg == NULL || strcmp(arg, "always") == 0) { + test_no_exec_ = 0; + } else if(strcmp(arg, "never") == 0) { + test_no_exec_ = 1; + } else if(strcmp(arg, "auto") == 0) { + /*noop*/ + } else { + fprintf(stderr, "%s: Unrecognized argument '%s' for option --exec.\n", test_argv0_, arg); + fprintf(stderr, "Try '%s --help' for more information.\n", test_argv0_); + test_exit_(2); + } + break; + + case 'E': + test_no_exec_ = 1; + break; + + case 't': +#if defined ACUTEST_WIN_ || defined ACUTEST_HAS_POSIX_TIMER_ + if(arg == NULL || strcmp(arg, "real") == 0) { + test_timer_ = 1; + #ifndef ACUTEST_WIN_ + } else if(strcmp(arg, "cpu") == 0) { + test_timer_ = 2; + #endif + } else { + fprintf(stderr, "%s: Unrecognized argument '%s' for option --time.\n", test_argv0_, arg); + fprintf(stderr, "Try '%s --help' for more information.\n", test_argv0_); + test_exit_(2); + } +#endif + break; + + case 'S': + test_no_summary_ = 1; + break; + + case 'T': + test_tap_ = 1; + break; + + case 'l': + test_list_names_(); + test_exit_(0); + break; + + case 'v': + test_verbose_level_ = (arg != NULL ? atoi(arg) : test_verbose_level_+1); + break; + + case 'q': + test_verbose_level_ = 0; + break; + + case 'c': + if(arg == NULL || strcmp(arg, "always") == 0) { + test_colorize_ = 1; + } else if(strcmp(arg, "never") == 0) { + test_colorize_ = 0; + } else if(strcmp(arg, "auto") == 0) { + /*noop*/ + } else { + fprintf(stderr, "%s: Unrecognized argument '%s' for option --color.\n", test_argv0_, arg); + fprintf(stderr, "Try '%s --help' for more information.\n", test_argv0_); + test_exit_(2); + } + break; + + case 'C': + test_colorize_ = 0; + break; + + case 'h': + test_help_(); + test_exit_(0); + break; + + case 'w': + test_worker_ = 1; + test_worker_index_ = atoi(arg); + break; + case 'x': + test_xml_output_ = fopen(arg, "w"); + if (!test_xml_output_) { + fprintf(stderr, "Unable to open '%s': %s\n", arg, strerror(errno)); + test_exit_(2); + } + break; + + case 0: + if(test_lookup_(arg) == 0) { + fprintf(stderr, "%s: Unrecognized unit test '%s'\n", test_argv0_, arg); + fprintf(stderr, "Try '%s --list' for list of unit tests.\n", test_argv0_); + test_exit_(2); + } + break; + + case TEST_CMDLINE_OPTID_UNKNOWN_: + fprintf(stderr, "Unrecognized command line option '%s'.\n", arg); + fprintf(stderr, "Try '%s --help' for more information.\n", test_argv0_); + test_exit_(2); + break; + + case TEST_CMDLINE_OPTID_MISSINGARG_: + fprintf(stderr, "The command line option '%s' requires an argument.\n", arg); + fprintf(stderr, "Try '%s --help' for more information.\n", test_argv0_); + test_exit_(2); + break; + + case TEST_CMDLINE_OPTID_BOGUSARG_: + fprintf(stderr, "The command line option '%s' does not expect an argument.\n", arg); + fprintf(stderr, "Try '%s --help' for more information.\n", test_argv0_); + test_exit_(2); + break; + } + + return 0; +} + + +#ifdef ACUTEST_LINUX_ +static int +test_is_tracer_present_(void) +{ + /* Must be large enough so the line 'TracerPid: ${PID}' can fit in. */ + static const int OVERLAP = 32; + + char buf[256+OVERLAP+1]; + int tracer_present = 0; + int fd; + size_t n_read = 0; + + fd = open("/proc/self/status", O_RDONLY); + if(fd == -1) + return 0; + + while(1) { + static const char pattern[] = "TracerPid:"; + const char* field; + + while(n_read < sizeof(buf) - 1) { + ssize_t n; + + n = read(fd, buf + n_read, sizeof(buf) - 1 - n_read); + if(n <= 0) + break; + n_read += n; + } + buf[n_read] = '\0'; + + field = strstr(buf, pattern); + if(field != NULL && field < buf + sizeof(buf) - OVERLAP) { + pid_t tracer_pid = (pid_t) atoi(field + sizeof(pattern) - 1); + tracer_present = (tracer_pid != 0); + break; + } + + if(n_read == sizeof(buf)-1) { + memmove(buf, buf + sizeof(buf)-1 - OVERLAP, OVERLAP); + n_read = OVERLAP; + } else { + break; + } + } + + close(fd); + return tracer_present; +} +#endif + +int +main(int argc, char** argv) +{ + int i; + test_argv0_ = argv[0]; + +#if defined ACUTEST_UNIX_ + test_colorize_ = isatty(STDOUT_FILENO); +#elif defined ACUTEST_WIN_ + #if defined _BORLANDC_ + test_colorize_ = isatty(_fileno(stdout)); + #else + test_colorize_ = _isatty(_fileno(stdout)); + #endif +#else + test_colorize_ = 0; +#endif + + /* Count all test units */ + test_list_size_ = 0; + for(i = 0; test_list_[i].func != NULL; i++) + test_list_size_++; + + test_details_ = (struct test_detail_*)calloc(test_list_size_, sizeof(struct test_detail_)); + if(test_details_ == NULL) { + fprintf(stderr, "Out of memory.\n"); + test_exit_(2); + } + + /* Parse options */ + test_cmdline_read_(test_cmdline_options_, argc, argv, test_cmdline_callback_); + + /* Initialize the proper timer. */ + test_timer_init_(); + +#if defined(ACUTEST_WIN_) + SetUnhandledExceptionFilter(test_seh_exception_filter_); +#ifdef _MSC_VER + _set_abort_behavior(0, _WRITE_ABORT_MSG); +#endif +#endif + + /* By default, we want to run all tests. */ + if(test_count_ == 0) { + for(i = 0; test_list_[i].func != NULL; i++) + test_remember_(i); + } + + /* Guess whether we want to run unit tests as child processes. */ + if(test_no_exec_ < 0) { + test_no_exec_ = 0; + + if(test_count_ <= 1) { + test_no_exec_ = 1; + } else { +#ifdef ACUTEST_WIN_ + if(IsDebuggerPresent()) + test_no_exec_ = 1; +#endif +#ifdef ACUTEST_LINUX_ + if(test_is_tracer_present_()) + test_no_exec_ = 1; +#endif +#ifdef RUNNING_ON_VALGRIND + /* RUNNING_ON_VALGRIND is provided by valgrind.h */ + if(RUNNING_ON_VALGRIND) + test_no_exec_ = 1; +#endif + } + } + + if(test_tap_) { + /* TAP requires we know test result ("ok", "not ok") before we output + * anything about the test, and this gets problematic for larger verbose + * levels. */ + if(test_verbose_level_ > 2) + test_verbose_level_ = 2; + + /* TAP harness should provide some summary. */ + test_no_summary_ = 1; + + if(!test_worker_) + printf("1..%d\n", (int) test_count_); + } + + int index = test_worker_index_; + for(i = 0; test_list_[i].func != NULL; i++) { + int run = (test_details_[i].flags & TEST_FLAG_RUN_); + if (test_skip_mode_) /* Run all tests except those listed. */ + run = !run; + if(run) + test_run_(&test_list_[i], index++, i); + } + + /* Write a summary */ + if(!test_no_summary_ && test_verbose_level_ >= 1) { + if(test_verbose_level_ >= 3) { + test_print_in_color_(TEST_COLOR_DEFAULT_INTENSIVE_, "Summary:\n"); + + printf(" Count of all unit tests: %4d\n", (int) test_list_size_); + printf(" Count of run unit tests: %4d\n", test_stat_run_units_); + printf(" Count of failed unit tests: %4d\n", test_stat_failed_units_); + printf(" Count of skipped unit tests: %4d\n", (int) test_list_size_ - test_stat_run_units_); + } + + if(test_stat_failed_units_ == 0) { + test_print_in_color_(TEST_COLOR_GREEN_INTENSIVE_, "SUCCESS:"); + printf(" All unit tests have passed.\n"); + } else { + test_print_in_color_(TEST_COLOR_RED_INTENSIVE_, "FAILED:"); + printf(" %d of %d unit tests %s failed.\n", + test_stat_failed_units_, test_stat_run_units_, + (test_stat_failed_units_ == 1) ? "has" : "have"); + } + + if(test_verbose_level_ >= 3) + printf("\n"); + } + + if (test_xml_output_) { +#if defined ACUTEST_UNIX_ + char *suite_name = basename(argv[0]); +#elif defined ACUTEST_WIN_ + char suite_name[_MAX_FNAME]; + _splitpath(argv[0], NULL, NULL, suite_name, NULL); +#else + const char *suite_name = argv[0]; +#endif + fprintf(test_xml_output_, "\n"); + fprintf(test_xml_output_, "\n", + suite_name, (int)test_list_size_, test_stat_failed_units_, test_stat_failed_units_, + (int)test_list_size_ - test_stat_run_units_); + for(i = 0; test_list_[i].func != NULL; i++) { + struct test_detail_ *details = &test_details_[i]; + fprintf(test_xml_output_, " \n", test_list_[i].name, details->duration); + if (details->flags & TEST_FLAG_FAILURE_) + fprintf(test_xml_output_, " \n"); + if (!(details->flags & TEST_FLAG_FAILURE_) && !(details->flags & TEST_FLAG_SUCCESS_)) + fprintf(test_xml_output_, " \n"); + fprintf(test_xml_output_, " \n"); + } + fprintf(test_xml_output_, "\n"); + fclose(test_xml_output_); + } + + test_cleanup_(); + + return (test_stat_failed_units_ == 0) ? 0 : 1; +} + + +#endif /* #ifndef TEST_NO_MAIN */ + +#ifdef _MSC_VER + #pragma warning(pop) +#endif + +#ifdef __cplusplus + } /* extern "C" */ +#endif + +#endif /* #ifndef ACUTEST_H */ diff --git a/tests/tests.cpp b/tests/tests.cpp new file mode 100644 index 00000000..98f7ee83 --- /dev/null +++ b/tests/tests.cpp @@ -0,0 +1,283 @@ +#include "acutest.h" + +#include "config.hpp" + +void +test_nop() +{ + TEST_CHECK(true); +} + +void +test_config_bool() +{ + ConfigBOOL v; + + TEST_CHECK(v.from_string("true") == 0); + TEST_CHECK(v == true); + TEST_CHECK(v.to_string() == "true"); + + TEST_CHECK(v.from_string("false") == 0); + TEST_CHECK(v == false); + TEST_CHECK(v.to_string() == "false"); + + TEST_CHECK(v.from_string("asdf") == -EINVAL); +} + +void +test_config_uint64() +{ + ConfigUINT64 v; + + TEST_CHECK(v.from_string("0") == 0); + TEST_CHECK(v == (uint64_t)0); + TEST_CHECK(v.to_string() == "0"); + + TEST_CHECK(v.from_string("123") == 0); + TEST_CHECK(v == (uint64_t)123); + TEST_CHECK(v.to_string() == "123"); + + TEST_CHECK(v.from_string("1234567890") == 0); + TEST_CHECK(v == (uint64_t)1234567890); + TEST_CHECK(v.to_string() == "1234567890"); + + TEST_CHECK(v.from_string("asdf") == -EINVAL); +} + +void +test_config_int() +{ + ConfigINT v; + + TEST_CHECK(v.from_string("0") == 0); + TEST_CHECK(v == (int)0); + TEST_CHECK(v.to_string() == "0"); + + TEST_CHECK(v.from_string("123") == 0); + TEST_CHECK(v == (int)123); + TEST_CHECK(v.to_string() == "123"); + + TEST_CHECK(v.from_string("1234567890") == 0); + TEST_CHECK(v == (int)1234567890); + TEST_CHECK(v.to_string() == "1234567890"); + + TEST_CHECK(v.from_string("asdf") == -EINVAL); +} + +void +test_config_str() +{ + ConfigSTR v; + + TEST_CHECK(v.from_string("foo") == 0); + TEST_CHECK(v == "foo"); + TEST_CHECK(v.to_string() == "foo"); +} + +void +test_config_branches() +{ + uint64_t minfreespace; + Branches b(minfreespace); + Branches::CPtr bcp0; + Branches::CPtr bcp1; + + minfreespace = 1234; + TEST_CHECK(b->minfreespace() == 1234); + TEST_CHECK(b.to_string() == ""); + + bcp0 = b; + TEST_CHECK(b.from_string("/foo/bar") == 0); + TEST_CHECK(b.to_string() == "/foo/bar=RW"); + bcp1 = b; + TEST_CHECK(bcp0.get() != bcp1.get()); + + TEST_CHECK(b.from_string("/foo/bar=RW,1234") == 0); + TEST_CHECK(b.to_string() == "/foo/bar=RW,1234"); + + TEST_CHECK(b.from_string("/foo/bar=RO,1234:/foo/baz=NC,4321") == 0); + TEST_CHECK(b.to_string() == "/foo/bar=RO,1234:/foo/baz=NC,4321"); + bcp0 = b; + TEST_CHECK((*bcp0).size() == 2); + TEST_CHECK((*bcp0)[0].path == "/foo/bar"); + TEST_CHECK((*bcp0)[0].minfreespace() == 1234); + TEST_MSG("minfreespace: expected = %lu; produced = %lu", + 1234, + (*bcp0)[0].minfreespace()); + TEST_CHECK((*bcp0)[1].path == "/foo/baz"); + TEST_CHECK((*bcp0)[1].minfreespace() == 4321); + TEST_MSG("minfreespace: expected = %lu; produced = %lu", + 4321, + (*bcp0)[1].minfreespace()); +} + +void +test_config_cachefiles() +{ + CacheFiles cf; + + TEST_CHECK(cf.from_string("libfuse") == 0); + TEST_CHECK(cf.to_string() == "libfuse"); + TEST_CHECK(cf.from_string("off") == 0); + TEST_CHECK(cf.to_string() == "off"); + TEST_CHECK(cf.from_string("partial") == 0); + TEST_CHECK(cf.to_string() == "partial"); + TEST_CHECK(cf.from_string("full") == 0); + TEST_CHECK(cf.to_string() == "full"); + TEST_CHECK(cf.from_string("auto-full") == 0); + TEST_CHECK(cf.to_string() == "auto-full"); + TEST_CHECK(cf.from_string("foobar") == -EINVAL); +} + +void +test_config_inodecalc() +{ + InodeCalc ic("passthrough"); + + TEST_CHECK(ic.from_string("passthrough") == 0); + TEST_CHECK(ic.to_string() == "passthrough"); + TEST_CHECK(ic.from_string("path-hash") == 0); + TEST_CHECK(ic.to_string() == "path-hash"); + TEST_CHECK(ic.from_string("devino-hash") == 0); + TEST_CHECK(ic.to_string() == "devino-hash"); + TEST_CHECK(ic.from_string("hybrid-hash") == 0); + TEST_CHECK(ic.to_string() == "hybrid-hash"); + TEST_CHECK(ic.from_string("path-hash32") == 0); + TEST_CHECK(ic.to_string() == "path-hash32"); + TEST_CHECK(ic.from_string("devino-hash32") == 0); + TEST_CHECK(ic.to_string() == "devino-hash32"); + TEST_CHECK(ic.from_string("hybrid-hash32") == 0); + TEST_CHECK(ic.to_string() == "hybrid-hash32"); + TEST_CHECK(ic.from_string("asdf") == -EINVAL); +} + +void +test_config_moveonenospc() +{ + MoveOnENOSPC m(false); + + TEST_CHECK(m.to_string() == "false"); + TEST_CHECK(m.from_string("mfs") == 0); + TEST_CHECK(m.to_string() == "mfs"); + TEST_CHECK(m.from_string("mspmfs") == 0); + TEST_CHECK(m.to_string() == "mspmfs"); + TEST_CHECK(m.from_string("true") == 0); + TEST_CHECK(m.to_string() == "mfs"); + TEST_CHECK(m.from_string("blah") == -EINVAL); +} + +void +test_config_nfsopenhack() +{ + NFSOpenHack n; + + TEST_CHECK(n.from_string("off") == 0); + TEST_CHECK(n.to_string() == "off"); + TEST_CHECK(n == NFSOpenHack::ENUM::OFF); + + TEST_CHECK(n.from_string("git") == 0); + TEST_CHECK(n.to_string() == "git"); + TEST_CHECK(n == NFSOpenHack::ENUM::GIT); + + TEST_CHECK(n.from_string("all") == 0); + TEST_CHECK(n.to_string() == "all"); + TEST_CHECK(n == NFSOpenHack::ENUM::ALL); +} + +void +test_config_readdir() +{ + ReadDir r; + + TEST_CHECK(r.from_string("linux") == 0); + TEST_CHECK(r.to_string() == "linux"); + TEST_CHECK(r == ReadDir::ENUM::LINUX); + + TEST_CHECK(r.from_string("posix") == 0); + TEST_CHECK(r.to_string() == "posix"); + TEST_CHECK(r == ReadDir::ENUM::POSIX); +} + +void +test_config_statfs() +{ + StatFS s; + + TEST_CHECK(s.from_string("base") == 0); + TEST_CHECK(s.to_string() == "base"); + TEST_CHECK(s == StatFS::ENUM::BASE); + + TEST_CHECK(s.from_string("full") == 0); + TEST_CHECK(s.to_string() == "full"); + TEST_CHECK(s == StatFS::ENUM::FULL); + + TEST_CHECK(s.from_string("asdf") == -EINVAL); +} + +void +test_config_statfs_ignore() +{ + StatFSIgnore s; + + TEST_CHECK(s.from_string("none") == 0); + TEST_CHECK(s.to_string() == "none"); + TEST_CHECK(s == StatFSIgnore::ENUM::NONE); + + TEST_CHECK(s.from_string("ro") == 0); + TEST_CHECK(s.to_string() == "ro"); + TEST_CHECK(s == StatFSIgnore::ENUM::RO); + + TEST_CHECK(s.from_string("nc") == 0); + TEST_CHECK(s.to_string() == "nc"); + TEST_CHECK(s == StatFSIgnore::ENUM::NC); + + TEST_CHECK(s.from_string("asdf") == -EINVAL); +} + +void +test_config_xattr() +{ + XAttr x; + + TEST_CHECK(x.from_string("passthrough") == 0); + TEST_CHECK(x.to_string() == "passthrough"); + TEST_CHECK(x == XAttr::ENUM::PASSTHROUGH); + + TEST_CHECK(x.from_string("noattr") == 0); + TEST_CHECK(x.to_string() == "noattr"); + TEST_CHECK(x == XAttr::ENUM::NOATTR); + + TEST_CHECK(x.from_string("nosys") == 0); + TEST_CHECK(x.to_string() == "nosys"); + TEST_CHECK(x == XAttr::ENUM::NOSYS); + + TEST_CHECK(x.from_string("asdf") == -EINVAL); +} + +void +test_config() +{ + Config cfg; + + TEST_CHECK(cfg.set_raw("async_read","true") == 0); +} + +TEST_LIST = + { + {"nop",test_nop}, + {"config_bool",test_config_bool}, + {"config_uint64",test_config_uint64}, + {"config_int",test_config_int}, + {"config_str",test_config_str}, + {"config_branches",test_config_branches}, + {"config_cachefiles",test_config_cachefiles}, + {"config_inodecalc",test_config_inodecalc}, + {"config_moveonenospc",test_config_moveonenospc}, + {"config_nfsopenhack",test_config_nfsopenhack}, + {"config_readdir",test_config_readdir}, + {"config_statfs",test_config_statfs}, + {"config_statfsignore",test_config_statfs_ignore}, + {"config_xattr",test_config_xattr}, + {"config",test_config}, + {NULL,NULL} + }; diff --git a/tools/cppfind b/tools/cppfind index 5721d365..51c99029 100755 --- a/tools/cppfind +++ b/tools/cppfind @@ -3,6 +3,6 @@ CXX="${CXX:-c++}" FUSE_CFLAGS="-Ilibfuse/include -D_FILE_OFFSET_BITS=64 -DFUSE_USE_VERSION=29" -echo "#include " | ${CXX} -E ${FUSE_CFLAGS} - | grep "${1}" > /dev/null +echo "#include \"fuse.h\"" | ${CXX} -E ${FUSE_CFLAGS} - | grep "${1}" > /dev/null [ "$?" != "0" ]; echo $? From dfb544fed892506634e5aa7ef472a3a6a44f7d6d Mon Sep 17 00:00:00 2001 From: Antonio SJ Musumeci Date: Sat, 23 Jan 2021 17:18:07 -0500 Subject: [PATCH 4/6] README: add warning/clearity about caching script and hardlinks and cow --- README.md | 13 +- man/mergerfs.1 | 1617 +++++++++++++++++++++++++----------------------- 2 files changed, 842 insertions(+), 788 deletions(-) diff --git a/README.md b/README.md index af27bce8..da0ebada 100644 --- a/README.md +++ b/README.md @@ -767,6 +767,9 @@ With #2 one could use dm-cache as well but there is another solution which requi Move files from cache to backing pool based only on the last time the file was accessed. Replace `-atime` with `-amin` if you want minutes rather than days. May want to use the `fadvise` / `--drop-cache` version of rsync or run rsync with the tool "nocache". +*NOTE:* The arguments to these scripts include the cache **drive**. Not the pool with the cache drive. You could have data loss if the source is the cache pool. + + ``` #!/bin/bash @@ -788,6 +791,8 @@ find "${CACHE}" -type f -atime +${N} -printf '%P\n' | \ Move the oldest file from the cache to the backing pool. Continue till below percentage threshold. +*NOTE:* The arguments to these scripts include the cache **drive**. Not the pool with the cache drive. You could have data loss if the source is the cache pool. + ``` #!/bin/bash @@ -1067,17 +1072,21 @@ The default create policy is `epmfs`. That is a path preserving algorithm. With This catches a lot of new users off guard but changing the default would break the setup for many existing users. If you do not care about path preservation and wish your files to be spread across all your drives change to `mfs` or similar policy as described above. If you do want path preservation you'll need to perform the manual act of creating paths on the drives you want the data to land on before transferring your data. Setting `func.mkdir=epall` can simplify managing path preservation for `create`. Or use `func.mkdir=rand` if you're interested in just grouping together directory content by drive. -#### Do hard links work? +#### Do hardlinks work? Yes. You need to use `use_ino` to support proper reporting of inodes but they work regardless. See also the option `inodecalc`. What mergerfs does not do is fake hard links across branches. Read the section "rename & link" for how it works. +Remember that hardlinks will NOT work across devices. That includes between the original filesystem and a mergerfs pool, between two separate pools of the same underlying filesystems, or bind mounts of paths within the mergerfs pool. The latter is common when using Docker or Podman. Multiple volumes (bind mounts) to the same underlying filesystem are considered different devices. There is no way to link between them. You should mount in the highest directory in the mergerfs pool that includes all the paths you need if you want links to work. -#### Does mergerfs support CoW / copy-on-write? + +#### Does mergerfs support CoW / copy-on-write / writes to read-only filesystems? Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs or aufs sense. It does offer a [cow-shell](http://manpages.ubuntu.com/manpages/bionic/man1/cow-shell.1.html) like hard link breaking (copy to temp file then rename over original) which can be useful when wanting to save space by hardlinking duplicate files but wish to treat each name as if it were a unique and separate file. +If you want to write to a read-only filesystem you should look at overlayfs. You can always include the overlayfs mount into a mergerfs pool. + #### Why can't I see my files / directories? diff --git a/man/mergerfs.1 b/man/mergerfs.1 index 9f79e2f5..5fa4e12a 100644 --- a/man/mergerfs.1 +++ b/man/mergerfs.1 @@ -1,5 +1,5 @@ .\"t -.\" Automatically generated by Pandoc 1.19.2.4 +.\" Automatically generated by Pandoc 2.5 .\" .TH "mergerfs" "1" "2020\-08\-30" "mergerfs user manual" "" .hy @@ -11,10 +11,11 @@ mergerfs \- a featureful union filesystem mergerfs \-o .SH DESCRIPTION .PP -\f[B]mergerfs\f[] is a union filesystem geared towards simplifying +\f[B]mergerfs\f[R] is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. -It is similar to \f[B]mhddfs\f[], \f[B]unionfs\f[], and \f[B]aufs\f[]. +It is similar to \f[B]mhddfs\f[R], \f[B]unionfs\f[R], and +\f[B]aufs\f[R]. .SH FEATURES .IP \[bu] 2 Runs in userspace (FUSE) @@ -53,33 +54,33 @@ Read more about policies below. .IP .nf \f[C] -A\ \ \ \ \ \ \ \ \ +\ \ \ \ \ \ B\ \ \ \ \ \ \ \ =\ \ \ \ \ \ \ C -/disk1\ \ \ \ \ \ \ \ \ \ \ /disk2\ \ \ \ \ \ \ \ \ \ \ /merged -|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | -+\-\-\ /dir1\ \ \ \ \ \ \ \ +\-\-\ /dir1\ \ \ \ \ \ \ \ +\-\-\ /dir1 -|\ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ | -|\ \ \ +\-\-\ file1\ \ \ \ |\ \ \ +\-\-\ file2\ \ \ \ |\ \ \ +\-\-\ file1 -|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file3\ \ \ \ |\ \ \ +\-\-\ file2 -+\-\-\ /dir2\ \ \ \ \ \ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file3 -|\ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ /dir3\ \ \ \ \ \ \ \ | -|\ \ \ +\-\-\ file4\ \ \ \ \ \ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ /dir2 -|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ file5\ \ \ |\ \ \ | -+\-\-\ file6\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file4 -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ /dir3 -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ | -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file5 -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ file6 -\f[] +A + B = C +/disk1 /disk2 /merged +| | | ++\-\- /dir1 +\-\- /dir1 +\-\- /dir1 +| | | | | | +| +\-\- file1 | +\-\- file2 | +\-\- file1 +| | +\-\- file3 | +\-\- file2 ++\-\- /dir2 | | +\-\- file3 +| | +\-\- /dir3 | +| +\-\- file4 | +\-\- /dir2 +| +\-\- file5 | | ++\-\- file6 | +\-\- file4 + | + +\-\- /dir3 + | | + | +\-\- file5 + | + +\-\- file6 +\f[R] .fi .PP -mergerfs does \f[B]NOT\f[] support the copy\-on\-write (CoW) or whiteout -behaviors found in \f[B]aufs\f[] and \f[B]overlayfs\f[]. -You can \f[B]not\f[] mount a read\-only filesystem and write to it. +mergerfs does \f[B]NOT\f[R] support the copy\-on\-write (CoW) or +whiteout behaviors found in \f[B]aufs\f[R] and \f[B]overlayfs\f[R]. +You can \f[B]not\f[R] mount a read\-only filesystem and write to it. However, mergerfs will ignore read\-only drives when creating new files so you can mix read\-write and read\-only drives. -It also does \f[B]NOT\f[] split data across drives. +It also does \f[B]NOT\f[R] split data across drives. It is not RAID0 / striping. It is simply a union of other filesystems. .SH TERMINOLOGY @@ -102,15 +103,14 @@ path preservation: Aspect of some policies which includes checking the path for which a file would be created. .SH BASIC SETUP .PP -If you don\[aq]t already know that you have a special use case then just +If you don\[cq]t already know that you have a special use case then just start with one of the following option sets. -.SS You need \f[C]mmap\f[] (used by rtorrent and many sqlite3 base -software) +.SS You need \f[C]mmap\f[R] (used by rtorrent and many sqlite3 base software) .PP -\f[C]allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs\f[] -.SS You don\[aq]t need \f[C]mmap\f[] +\f[C]allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs\f[R] +.SS You don\[cq]t need \f[C]mmap\f[R] .PP -\f[C]allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs\f[] +\f[C]allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs\f[R] .PP See the mergerfs wiki for real world deployments (https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments) @@ -118,127 +118,125 @@ for comparisons / ideas. .SH OPTIONS .PP These options are the same regardless you use them with the -\f[C]mergerfs\f[] commandline program, used in fstab, or in a config +\f[C]mergerfs\f[R] commandline program, used in fstab, or in a config file. .SS mount options .IP \[bu] 2 -\f[B]config\f[]: Path to a config file. +\f[B]config\f[R]: Path to a config file. Same arguments as below in key=val / ini style format. .IP \[bu] 2 -\f[B]branches\f[]: Colon delimited list of branches. +\f[B]branches\f[R]: Colon delimited list of branches. .IP \[bu] 2 -\f[B]allow_other\f[]: A libfuse option which allows users besides the +\f[B]allow_other\f[R]: A libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use\-cases. .IP \[bu] 2 -\f[B]minfreespace=SIZE\f[]: The minimum space value used for creation +\f[B]minfreespace=SIZE\f[R]: The minimum space value used for creation policies. Can be overridden by branch specific option. -Understands \[aq]K\[aq], \[aq]M\[aq], and \[aq]G\[aq] to represent -kilobyte, megabyte, and gigabyte respectively. +Understands `K', `M', and `G' to represent kilobyte, megabyte, and +gigabyte respectively. (default: 4G) .IP \[bu] 2 -\f[B]moveonenospc=BOOL|POLICY\f[]: When enabled if a \f[B]write\f[] -fails with \f[B]ENOSPC\f[] (no space left on device) or \f[B]EDQUOT\f[] -(disk quota exceeded) the policy selected will run to find a new -location for the file. +\f[B]moveonenospc=BOOL|POLICY\f[R]: When enabled if a \f[B]write\f[R] +fails with \f[B]ENOSPC\f[R] (no space left on device) or +\f[B]EDQUOT\f[R] (disk quota exceeded) the policy selected will run to +find a new location for the file. An attempt to move the file to that branch will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false, true = mfs) .IP \[bu] 2 -\f[B]use_ino\f[]: Causes mergerfs to supply file/directory inodes rather -than libfuse. +\f[B]use_ino\f[R]: Causes mergerfs to supply file/directory inodes +rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value. .IP \[bu] 2 -\f[B]inodecalc=passthrough|path\-hash|devino\-hash|hybrid\-hash\f[]: +\f[B]inodecalc=passthrough|path\-hash|devino\-hash|hybrid\-hash\f[R]: Selects the inode calculation algorithm. (default: hybrid\-hash) .IP \[bu] 2 -\f[B]dropcacheonclose=BOOL\f[]: When a file is requested to be closed -call \f[C]posix_fadvise\f[] on it first to instruct the kernel that we +\f[B]dropcacheonclose=BOOL\f[R]: When a file is requested to be closed +call \f[C]posix_fadvise\f[R] on it first to instruct the kernel that we no longer need the data and it can drop its cache. -Recommended when \f[B]cache.files=partial|full|auto\-full\f[] to limit +Recommended when \f[B]cache.files=partial|full|auto\-full\f[R] to limit double caching. (default: false) .IP \[bu] 2 -\f[B]symlinkify=BOOL\f[]: When enabled and a file is not writable and -its mtime or ctime is older than \f[B]symlinkify_timeout\f[] files will +\f[B]symlinkify=BOOL\f[R]: When enabled and a file is not writable and +its mtime or ctime is older than \f[B]symlinkify_timeout\f[R] files will be reported as symlinks to the original files. Please read more below before using. (default: false) .IP \[bu] 2 -\f[B]symlinkify_timeout=UINT\f[]: Time to wait, in seconds, to activate -the \f[B]symlinkify\f[] behavior. +\f[B]symlinkify_timeout=UINT\f[R]: Time to wait, in seconds, to activate +the \f[B]symlinkify\f[R] behavior. (default: 3600) .IP \[bu] 2 -\f[B]nullrw=BOOL\f[]: Turns reads and writes into no\-ops. +\f[B]nullrw=BOOL\f[R]: Turns reads and writes into no\-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false) .IP \[bu] 2 -\f[B]ignorepponrename=BOOL\f[]: Ignore path preserving on rename. +\f[B]ignorepponrename=BOOL\f[R]: Ignore path preserving on rename. Typically rename and link act differently depending on the policy of -\f[C]create\f[] (read below). +\f[C]create\f[R] (read below). Enabling this will cause rename and link to always use the non\-path preserving behavior. This means files, when renamed or linked, will stay on the same drive. (default: false) .IP \[bu] 2 -\f[B]security_capability=BOOL\f[]: If false return ENOATTR when xattr +\f[B]security_capability=BOOL\f[R]: If false return ENOATTR when xattr security.capability is queried. (default: true) .IP \[bu] 2 -\f[B]xattr=passthrough|noattr|nosys\f[]: Runtime control of xattrs. +\f[B]xattr=passthrough|noattr|nosys\f[R]: Runtime control of xattrs. Default is to passthrough xattr requests. -\[aq]noattr\[aq] will short circuit as if nothing exists. -\[aq]nosys\[aq] will respond with ENOSYS as if xattrs are not supported -or disabled. +`noattr' will short circuit as if nothing exists. +`nosys' will respond with ENOSYS as if xattrs are not supported or +disabled. (default: passthrough) .IP \[bu] 2 -\f[B]link_cow=BOOL\f[]: When enabled if a regular file is opened which +\f[B]link_cow=BOOL\f[R]: When enabled if a regular file is opened which has a link count > 1 it will copy the file to a temporary file and rename over the original. Breaking the link and providing a basic copy\-on\-write function similar to cow\-shell. (default: false) .IP \[bu] 2 -\f[B]statfs=base|full\f[]: Controls how statfs works. -\[aq]base\[aq] means it will always use all branches in statfs -calculations. -\[aq]full\[aq] is in effect path preserving and only includes drives -where the path exists. +\f[B]statfs=base|full\f[R]: Controls how statfs works. +`base' means it will always use all branches in statfs calculations. +`full' is in effect path preserving and only includes drives where the +path exists. (default: base) .IP \[bu] 2 -\f[B]statfs_ignore=none|ro|nc\f[]: \[aq]ro\[aq] will cause statfs -calculations to ignore available space for branches mounted or tagged as -\[aq]read\-only\[aq] or \[aq]no create\[aq]. -\[aq]nc\[aq] will ignore available space for branches tagged as \[aq]no -create\[aq]. +\f[B]statfs_ignore=none|ro|nc\f[R]: `ro' will cause statfs calculations +to ignore available space for branches mounted or tagged as `read\-only' +or `no create'. +`nc' will ignore available space for branches tagged as `no create'. (default: none) .IP \[bu] 2 -\f[B]nfsopenhack=off|git|all\f[]: A workaround for exporting mergerfs +\f[B]nfsopenhack=off|git|all\f[R]: A workaround for exporting mergerfs over NFS where there are issues with creating files for write while setting the mode to read\-only. (default: off) .IP \[bu] 2 -\f[B]posix_acl=BOOL\f[]: Enable POSIX ACL support (if supported by +\f[B]posix_acl=BOOL\f[R]: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false) .IP \[bu] 2 -\f[B]async_read=BOOL\f[]: Perform reads asynchronously. +\f[B]async_read=BOOL\f[R]: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true) .IP \[bu] 2 -\f[B]fuse_msg_size=UINT\f[]: Set the max number of pages per FUSE +\f[B]fuse_msg_size=UINT\f[R]: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256) .IP \[bu] 2 -\f[B]threads=INT\f[]: Number of threads to use in multithreaded mode. +\f[B]threads=INT\f[R]: Number of threads to use in multithreaded mode. When set to zero it will attempt to discover and use the number of logical cores. If the lookup fails it will fall back to using 4. @@ -251,90 +249,89 @@ NOTE: higher number of threads increases parallelism but usually decreases throughput. (default: 0) .IP \[bu] 2 -\f[B]fsname=STR\f[]: Sets the name of the filesystem as seen in -\f[B]mount\f[], \f[B]df\f[], etc. +\f[B]fsname=STR\f[R]: Sets the name of the filesystem as seen in +\f[B]mount\f[R], \f[B]df\f[R], etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed. .IP \[bu] 2 -\f[B]func.FUNC=POLICY\f[]: Sets the specific FUSE function\[aq]s policy. +\f[B]func.FUNC=POLICY\f[R]: Sets the specific FUSE function\[cq]s +policy. See below for the list of value types. -Example: \f[B]func.getattr=newest\f[] +Example: \f[B]func.getattr=newest\f[R] .IP \[bu] 2 -\f[B]category.action=POLICY\f[]: Sets policy of all FUSE functions in +\f[B]category.action=POLICY\f[R]: Sets policy of all FUSE functions in the action category. (default: epall) .IP \[bu] 2 -\f[B]category.create=POLICY\f[]: Sets policy of all FUSE functions in +\f[B]category.create=POLICY\f[R]: Sets policy of all FUSE functions in the create category. (default: epmfs) .IP \[bu] 2 -\f[B]category.search=POLICY\f[]: Sets policy of all FUSE functions in +\f[B]category.search=POLICY\f[R]: Sets policy of all FUSE functions in the search category. (default: ff) .IP \[bu] 2 -\f[B]cache.open=UINT\f[]: \[aq]open\[aq] policy cache timeout in -seconds. +\f[B]cache.open=UINT\f[R]: `open' policy cache timeout in seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.statfs=UINT\f[]: \[aq]statfs\[aq] cache timeout in seconds. +\f[B]cache.statfs=UINT\f[R]: `statfs' cache timeout in seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.attr=UINT\f[]: File attribute cache timeout in seconds. +\f[B]cache.attr=UINT\f[R]: File attribute cache timeout in seconds. (default: 1) .IP \[bu] 2 -\f[B]cache.entry=UINT\f[]: File name lookup cache timeout in seconds. +\f[B]cache.entry=UINT\f[R]: File name lookup cache timeout in seconds. (default: 1) .IP \[bu] 2 -\f[B]cache.negative_entry=UINT\f[]: Negative file name lookup cache +\f[B]cache.negative_entry=UINT\f[R]: Negative file name lookup cache timeout in seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.files=libfuse|off|partial|full|auto\-full\f[]: File page +\f[B]cache.files=libfuse|off|partial|full|auto\-full\f[R]: File page caching mode (default: libfuse) .IP \[bu] 2 -\f[B]cache.writeback=BOOL\f[]: Enable kernel writeback caching (default: -false) +\f[B]cache.writeback=BOOL\f[R]: Enable kernel writeback caching +(default: false) .IP \[bu] 2 -\f[B]cache.symlinks=BOOL\f[]: Cache symlinks (if supported by kernel) +\f[B]cache.symlinks=BOOL\f[R]: Cache symlinks (if supported by kernel) (default: false) .IP \[bu] 2 -\f[B]cache.readdir=BOOL\f[]: Cache readdir (if supported by kernel) +\f[B]cache.readdir=BOOL\f[R]: Cache readdir (if supported by kernel) (default: false) .IP \[bu] 2 -\f[B]direct_io\f[]: deprecated \- Bypass page cache. -Use \f[C]cache.files=off\f[] instead. +\f[B]direct_io\f[R]: deprecated \- Bypass page cache. +Use \f[C]cache.files=off\f[R] instead. (default: false) .IP \[bu] 2 -\f[B]kernel_cache\f[]: deprecated \- Do not invalidate data cache on +\f[B]kernel_cache\f[R]: deprecated \- Do not invalidate data cache on file open. -Use \f[C]cache.files=full\f[] instead. +Use \f[C]cache.files=full\f[R] instead. (default: false) .IP \[bu] 2 -\f[B]auto_cache\f[]: deprecated \- Invalidate data cache if file mtime +\f[B]auto_cache\f[R]: deprecated \- Invalidate data cache if file mtime or size change. -Use \f[C]cache.files=auto\-full\f[] instead. +Use \f[C]cache.files=auto\-full\f[R] instead. (default: false) .IP \[bu] 2 -\f[B]async_read\f[]: deprecated \- Perform reads asynchronously. -Use \f[C]async_read=true\f[] instead. +\f[B]async_read\f[R]: deprecated \- Perform reads asynchronously. +Use \f[C]async_read=true\f[R] instead. .IP \[bu] 2 -\f[B]sync_read\f[]: deprecated \- Perform reads synchronously. -Use \f[C]async_read=false\f[] instead. +\f[B]sync_read\f[R]: deprecated \- Perform reads synchronously. +Use \f[C]async_read=false\f[R] instead. .PP -\f[B]NOTE:\f[] Options are evaluated in the order listed so if the -options are \f[B]func.rmdir=rand,category.action=ff\f[] the -\f[B]action\f[] category setting will override the \f[B]rmdir\f[] +\f[B]NOTE:\f[R] Options are evaluated in the order listed so if the +options are \f[B]func.rmdir=rand,category.action=ff\f[R] the +\f[B]action\f[R] category setting will override the \f[B]rmdir\f[R] setting. .SS Value Types .IP \[bu] 2 -BOOL = \[aq]true\[aq] | \[aq]false\[aq] +BOOL = `true' | `false' .IP \[bu] 2 INT = [MIN_INT,MAX_INT] .IP \[bu] 2 UINT = [0,MAX_INT] .IP \[bu] 2 -SIZE = \[aq]NNM\[aq]; NN = INT, M = \[aq]K\[aq] | \[aq]M\[aq] | -\[aq]G\[aq] | \[aq]T\[aq] +SIZE = `NNM'; NN = INT, M = `K' | `M' | `G' | `T' .IP \[bu] 2 STR = string .IP \[bu] 2 @@ -345,19 +342,19 @@ CATEGORY = function category POLICY = mergerfs function policy .SS branches .PP -The \[aq]branches\[aq] argument is a colon (\[aq]:\[aq]) delimited list -of paths to be pooled together. +The `branches' argument is a colon (`:') delimited list of paths to be +pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same -device and any features which aren\[aq]t supported by the underlying +device and any features which aren\[cq]t supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors. .PP Branches currently have two options which can be set. A type which impacts whether or not the branch is included in a policy calculation and a individual minfreespace value. -The values are set by prepending an \f[C]=\f[] at the end of a branch +The values are set by prepending an \f[C]=\f[R] at the end of a branch designation and using commas as delimiters. Example: /mnt/drive=RW,1234 .SS branch type @@ -365,13 +362,13 @@ Example: /mnt/drive=RW,1234 RW: (read/write) \- Default behavior. Will be eligible in all policy categories. .IP \[bu] 2 -RO: (read\-only) \- Will be excluded from \f[C]create\f[] and -\f[C]action\f[] policies. +RO: (read\-only) \- Will be excluded from \f[C]create\f[R] and +\f[C]action\f[R] policies. Same as a read\-only mounted filesystem would be (though faster to process). .IP \[bu] 2 -NC: (no\-create) \- Will be excluded from \f[C]create\f[] policies. -You can\[aq]t create on that branch but you can change or delete. +NC: (no\-create) \- Will be excluded from \f[C]create\f[R] policies. +You can\[cq]t create on that branch but you can change or delete. .SS minfreespace .PP Same purpose as the global option but specific to the branch. @@ -381,35 +378,35 @@ If not set the global value is used. To make it easier to include multiple branches mergerfs supports globbing (http://linux.die.net/man/7/glob). \f[B]The globbing tokens MUST be escaped when using via the shell else -the shell itself will apply the glob itself.\f[] +the shell itself will apply the glob itself.\f[R] .IP .nf \f[C] -#\ mergerfs\ \-o\ allow_other,use_ino\ /mnt/disk\\*:/mnt/cdrom\ /media/drives -\f[] +# mergerfs \-o allow_other,use_ino /mnt/disk\[rs]*:/mnt/cdrom /media/drives +\f[R] .fi .PP The above line will use all mount points in /mnt prefixed with -\f[B]disk\f[] and the \f[B]cdrom\f[]. +\f[B]disk\f[R] and the \f[B]cdrom\f[R]. .PP To have the pool mounted at boot or otherwise accessible from related -tools use \f[B]/etc/fstab\f[]. +tools use \f[B]/etc/fstab\f[R]. .IP .nf \f[C] -#\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ -/mnt/disk*:/mnt/cdrom\ \ /mnt/pool\ \ \ \ \ \ fuse.mergerfs\ \ allow_other,use_ino\ \ \ 0\ \ \ \ \ \ \ 0 -\f[] +# +/mnt/disk*:/mnt/cdrom /mnt/pool fuse.mergerfs allow_other,use_ino 0 0 +\f[R] .fi .PP -\f[B]NOTE:\f[] the globbing is done at mount or when updated using the +\f[B]NOTE:\f[R] the globbing is done at mount or when updated using the runtime API. If a new directory is added matching the glob after the fact it will not be automatically included. .PP -\f[B]NOTE:\f[] for mounting via \f[B]fstab\f[] to work you must have -\f[B]mount.fuse\f[] installed. -For Ubuntu/Debian it is included in the \f[B]fuse\f[] package. +\f[B]NOTE:\f[R] for mounting via \f[B]fstab\f[R] to work you must have +\f[B]mount.fuse\f[R] installed. +For Ubuntu/Debian it is included in the \f[B]fuse\f[R] package. .SS inodecalc .PP Inodes (st_ino) are unique identifiers within a filesystem. @@ -423,14 +420,14 @@ the complexity they add. .PP FUSE allows the server (mergerfs) to set inode values but not device IDs. -Creating an inode value is somewhat complex in mergerfs\[aq] case as -files aren\[aq]t really in its control. +Creating an inode value is somewhat complex in mergerfs\[cq] case as +files aren\[cq]t really in its control. If a policy changes what directory or file is to be selected or something changes out of band it becomes unclear what value should be used. Most software does not to care what the values are but those that do often break if a value changes unexpectedly. -The tool \f[C]find\f[] will abort a directory walk if it sees a +The tool \f[C]find\f[R] will abort a directory walk if it sees a directory inode change. NFS will return stale handle errors if the inode changes out of band. File dedup tools will usually leverage device ids and inodes as a @@ -446,27 +443,27 @@ problems mentioned above and could confuse file deduplication software as inodes from different filesystems can be the same. .IP \[bu] 2 path\-hash: Hashes the relative path of the entry in question. -The underlying file\[aq]s values are completely ignored. +The underlying file\[cq]s values are completely ignored. This means the inode value will always be the same for that file path. This is useful when using NFS and you make changes out of band such as copy data between branches. This also means that entries that do point to the same file will not be recognizable via inodes. -That \f[B]does not\f[] mean hard links don\[aq]t work. +That \f[B]does not\f[R] mean hard links don\[cq]t work. They will. .IP \[bu] 2 path\-hash32: 32bit version of path\-hash. .IP \[bu] 2 devino\-hash: Hashes the device id and inode of the underlying entry. -This won\[aq]t prevent issues with NFS should the policy pick a +This won\[cq]t prevent issues with NFS should the policy pick a different file or files move out of band but will present the same inode for underlying files that do too. .IP \[bu] 2 devino\-hash32: 32bit version of devino\-hash. .IP \[bu] 2 -hybrid\-hash: Performs \f[C]path\-hash\f[] on directories and -\f[C]devino\-hash\f[] on other file types. -Since directories can\[aq]t have hard links the static value won\[aq]t +hybrid\-hash: Performs \f[C]path\-hash\f[R] on directories and +\f[C]devino\-hash\f[R] on other file types. +Since directories can\[cq]t have hard links the static value won\[cq]t make a difference and the files will get values useful for finding duplicates. Probably the best to use if not using NFS. @@ -483,36 +480,36 @@ Unlike a typical filesystem FUSE filesystems can reuse inodes and not refer to the same entry. The internal identifier used to reference a file in FUSE is different from the inode value presented. -The former is the \f[C]nodeid\f[] and is actually a tuple of 2 64bit -values: \f[C]nodeid\f[] and \f[C]generation\f[]. +The former is the \f[C]nodeid\f[R] and is actually a tuple of 2 64bit +values: \f[C]nodeid\f[R] and \f[C]generation\f[R]. This tuple is not client facing. The inode that is presented to the client is passed through the kernel uninterpreted. .PP -From FUSE docs regarding \f[C]use_ino\f[]: +From FUSE docs regarding \f[C]use_ino\f[R]: .IP .nf \f[C] -Honor\ the\ st_ino\ field\ in\ the\ functions\ getattr()\ and -fill_dir().\ This\ value\ is\ used\ to\ fill\ in\ the\ st_ino\ field -in\ the\ stat(2),\ lstat(2),\ fstat(2)\ functions\ and\ the\ d_ino -field\ in\ the\ readdir(2)\ function.\ The\ filesystem\ does\ not -have\ to\ guarantee\ uniqueness,\ however\ some\ applications -rely\ on\ this\ value\ being\ unique\ for\ the\ whole\ filesystem. -Note\ that\ this\ does\ *not*\ affect\ the\ inode\ that\ libfuse -and\ the\ kernel\ use\ internally\ (also\ called\ the\ "nodeid"). -\f[] +Honor the st_ino field in the functions getattr() and +fill_dir(). This value is used to fill in the st_ino field +in the stat(2), lstat(2), fstat(2) functions and the d_ino +field in the readdir(2) function. The filesystem does not +have to guarantee uniqueness, however some applications +rely on this value being unique for the whole filesystem. +Note that this does *not* affect the inode that libfuse +and the kernel use internally (also called the \[dq]nodeid\[dq]). +\f[R] .fi .PP -In the future the \f[C]use_ino\f[] option will probably be removed as +In the future the \f[C]use_ino\f[R] option will probably be removed as this feature should replace the original libfuse inode calculation strategy. -Currently you still need to use \f[C]use_ino\f[] in order to enable -\f[C]inodecalc\f[]. +Currently you still need to use \f[C]use_ino\f[R] in order to enable +\f[C]inodecalc\f[R]. .SS fuse_msg_size .PP FUSE applications communicate with the kernel over a special character -device: \f[C]/dev/fuse\f[]. +device: \f[C]/dev/fuse\f[R]. A large portion of the overhead associated with FUSE is the cost of going back and forth from user space and kernel space over that device. Generally speaking the fewer trips needed the better the performance @@ -527,16 +524,16 @@ In Linux 4.20 a new feature was added allowing the negotiation of the max message size. Since the size is in multiples of pages (https://en.wikipedia.org/wiki/Page_(computer_memory)) the feature -is called \f[C]max_pages\f[]. -There is a maximum \f[C]max_pages\f[] value of 256 (1MiB) and minimum of -1 (4KiB). +is called \f[C]max_pages\f[R]. +There is a maximum \f[C]max_pages\f[R] value of 256 (1MiB) and minimum +of 1 (4KiB). The default used by Linux >=4.20, and hardcoded value used before 4.20, is 32 (128KiB). -In mergerfs its referred to as \f[C]fuse_msg_size\f[] to make it clear +In mergerfs its referred to as \f[C]fuse_msg_size\f[R] to make it clear what it impacts and provide some abstraction. .PP -Since there should be no downsides to increasing \f[C]fuse_msg_size\f[] -/ \f[C]max_pages\f[], outside a minor bump in RAM usage due to larger +Since there should be no downsides to increasing \f[C]fuse_msg_size\f[R] +/ \f[C]max_pages\f[R], outside a minor bump in RAM usage due to larger message buffers, mergerfs defaults the value to 256. On kernels before 4.20 the value has no effect. The reason the value is configurable is to enable experimentation and @@ -548,16 +545,16 @@ Due to the levels of indirection introduced by mergerfs and the underlying technology FUSE there can be varying levels of performance degradation. This feature will turn non\-directories which are not writable into -symlinks to the original file found by the \f[C]readlink\f[] policy +symlinks to the original file found by the \f[C]readlink\f[R] policy after the mtime and ctime are older than the timeout. .PP -\f[B]WARNING:\f[] The current implementation has a known issue in which +\f[B]WARNING:\f[R] The current implementation has a known issue in which if the file is open and being used when the file is converted to a symlink then the application which has that file open will receive an error when using it. This is unlikely to occur in practice but is something to keep in mind. .PP -\f[B]WARNING:\f[] Some backup solutions, such as CrashPlan, do not +\f[B]WARNING:\f[R] Some backup solutions, such as CrashPlan, do not backup the target of a symlink. If using this feature it will be necessary to point any backup software to the original drives or configure the software to follow symlinks if @@ -567,46 +564,46 @@ One for backup and one for general consumption. .SS nullrw .PP Due to how FUSE works there is an overhead to all requests made to a -FUSE filesystem that wouldn\[aq]t exist for an in kernel one. +FUSE filesystem that wouldn\[cq]t exist for an in kernel one. Meaning that even a simple passthrough will have some slowdown. However, generally the overhead is minimal in comparison to the cost of the underlying I/O. By disabling the underlying I/O we can test the theoretical performance boundaries. .PP -By enabling \f[C]nullrw\f[] mergerfs will work as it always does -\f[B]except\f[] that all reads and writes will be no\-ops. +By enabling \f[C]nullrw\f[R] mergerfs will work as it always does +\f[B]except\f[R] that all reads and writes will be no\-ops. A write will succeed (the size of the write will be returned as if it were successful) but mergerfs does nothing with the data it was given. -Similarly a read will return the size requested but won\[aq]t touch the +Similarly a read will return the size requested but won\[cq]t touch the buffer. .PP See the BENCHMARKING section for suggestions on how to test. .SS xattr .PP -Runtime extended attribute support can be managed via the \f[C]xattr\f[] -option. +Runtime extended attribute support can be managed via the +\f[C]xattr\f[R] option. By default it will passthrough any xattr calls. Given xattr support is rarely used and can have significant performance implications mergerfs allows it to be disabled at runtime. The performance problems mostly comes when file caching is enabled. -The kernel will send a \f[C]getxattr\f[] for -\f[C]security.capability\f[] \f[I]before every single write\f[]. -It doesn\[aq]t cache the responses to any \f[C]getxattr\f[]. +The kernel will send a \f[C]getxattr\f[R] for +\f[C]security.capability\f[R] \f[I]before every single write\f[R]. +It doesn\[cq]t cache the responses to any \f[C]getxattr\f[R]. This might be addressed in the future but for now mergerfs can really only offer the following workarounds. .PP -\f[C]noattr\f[] will cause mergerfs to short circuit all xattr calls and -return ENOATTR where appropriate. +\f[C]noattr\f[R] will cause mergerfs to short circuit all xattr calls +and return ENOATTR where appropriate. mergerfs still gets all the requests but they will not be forwarded on to the underlying filesystems. The runtime control will still function in this mode. .PP -\f[C]nosys\f[] will cause mergerfs to return ENOSYS for any xattr call. -The difference with \f[C]noattr\f[] is that the kernel will cache this +\f[C]nosys\f[R] will cause mergerfs to return ENOSYS for any xattr call. +The difference with \f[C]noattr\f[R] is that the kernel will cache this fact and itself short circuit future calls. -This is more efficient than \f[C]noattr\f[] but will cause mergerfs\[aq] -runtime control via the hidden file to stop working. +This is more efficient than \f[C]noattr\f[R] but will cause +mergerfs\[cq] runtime control via the hidden file to stop working. .SS nfsopenhack .PP NFS is not fully POSIX compliant and historically certain behaviors, @@ -623,53 +620,53 @@ of the NFS server and clients but it results in a permission error because a normal user is not allowed to open a read\-only file as writable. .PP -Even though it\[aq]s a more niche situation this hack breaks normal -security and behavior and as such is \f[C]off\f[] by default. -If set to \f[C]git\f[] it will only perform the hack when the path in -question includes \f[C]/.git/\f[]. -\f[C]all\f[] will result it it applying anytime a readonly file which is -empty is opened for writing. +Even though it\[cq]s a more niche situation this hack breaks normal +security and behavior and as such is \f[C]off\f[R] by default. +If set to \f[C]git\f[R] it will only perform the hack when the path in +question includes \f[C]/.git/\f[R]. +\f[C]all\f[R] will result it it applying anytime a readonly file which +is empty is opened for writing. .SH FUNCTIONS, CATEGORIES and POLICIES .PP The POSIX filesystem API is made up of a number of functions. -\f[B]creat\f[], \f[B]stat\f[], \f[B]chown\f[], etc. +\f[B]creat\f[R], \f[B]stat\f[R], \f[B]chown\f[R], etc. For ease of configuration in mergerfs most of the core functions are -grouped into 3 categories: \f[B]action\f[], \f[B]create\f[], and -\f[B]search\f[]. +grouped into 3 categories: \f[B]action\f[R], \f[B]create\f[R], and +\f[B]search\f[R]. These functions and categories can be assigned a policy which dictates which branch is chosen when performing that function. .PP -Some functions, listed in the category \f[C]N/A\f[] below, can not be +Some functions, listed in the category \f[C]N/A\f[R] below, can not be assigned the normal policies. These functions work with file handles, rather than file paths, which -were created by \f[C]open\f[] or \f[C]create\f[]. +were created by \f[C]open\f[R] or \f[C]create\f[R]. That said many times the current FUSE kernel driver will not always -provide the file handle when a client calls \f[C]fgetattr\f[], -\f[C]fchown\f[], \f[C]fchmod\f[], \f[C]futimens\f[], \f[C]ftruncate\f[], -etc. +provide the file handle when a client calls \f[C]fgetattr\f[R], +\f[C]fchown\f[R], \f[C]fchmod\f[R], \f[C]futimens\f[R], +\f[C]ftruncate\f[R], etc. This means it will call the regular, path based, versions. -\f[C]readdir\f[] has no real need for a policy given the purpose is +\f[C]readdir\f[R] has no real need for a policy given the purpose is merely to return a list of entries in a directory. -\f[C]statfs\f[]\[aq]s behavior can be modified via other options. +\f[C]statfs\f[R]\[cq]s behavior can be modified via other options. .PP -When using policies which are based on a branch\[aq]s available space +When using policies which are based on a branch\[cq]s available space the base path provided is used. Not the full path to the file in question. -Meaning that mounts in the branch won\[aq]t be considered in the space +Meaning that mounts in the branch won\[cq]t be considered in the space calculations. -The reason is that it doesn\[aq]t really work for non\-path preserving +The reason is that it doesn\[cq]t really work for non\-path preserving policies and can lead to non\-obvious behaviors. .PP NOTE: While any policy can be assigned to a function or category though some may not be very useful in practice. -For instance: \f[B]rand\f[] (random) may be useful for file creation -(create) but could lead to very odd behavior if used for \f[C]chmod\f[] +For instance: \f[B]rand\f[R] (random) may be useful for file creation +(create) but could lead to very odd behavior if used for \f[C]chmod\f[R] if there were more than one copy of the file. .SS Functions and their Category classifications .PP .TS tab(@); -lw(7.9n) lw(62.1n). +lw(7.4n) lw(62.6n). T{ Category T}@T{ @@ -702,52 +699,54 @@ T} .TE .PP In cases where something may be searched for (such as a path to clone) -\f[B]getattr\f[] will usually be used. +\f[B]getattr\f[R] will usually be used. .SS Policies .PP A policy is the algorithm used to choose a branch or branches for a function to work on. Think of them as ways to filter and sort branches. .PP -Any function in the \f[C]create\f[] category will clone the relative +Any function in the \f[C]create\f[R] category will clone the relative path if needed. -Some other functions (\f[C]rename\f[],\f[C]link\f[],\f[C]ioctl\f[]) have -special requirements or behaviors which you can read more about below. +Some other functions (\f[C]rename\f[R],\f[C]link\f[R],\f[C]ioctl\f[R]) +have special requirements or behaviors which you can read more about +below. .SS Filtering .PP Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting the branches. -Filters include \f[B]minfreespace\f[], whether or not a branch is +Filters include \f[B]minfreespace\f[R], whether or not a branch is mounted read\-only, and the branch tagging (RO,NC,RW). These filters are applied across all policies unless otherwise noted. .IP \[bu] 2 -No \f[B]search\f[] function policies filter. +No \f[B]search\f[R] function policies filter. .IP \[bu] 2 -All \f[B]action\f[] function policies filter out branches which are -mounted \f[B]read\-only\f[] or tagged as \f[B]RO (read\-only)\f[]. +All \f[B]action\f[R] function policies filter out branches which are +mounted \f[B]read\-only\f[R] or tagged as \f[B]RO (read\-only)\f[R]. .IP \[bu] 2 -All \f[B]create\f[] function policies filter out branches which are -mounted \f[B]read\-only\f[], tagged \f[B]RO (read\-only)\f[] or \f[B]NC -(no create)\f[], or has available space less than \f[C]minfreespace\f[]. +All \f[B]create\f[R] function policies filter out branches which are +mounted \f[B]read\-only\f[R], tagged \f[B]RO (read\-only)\f[R] or +\f[B]NC (no create)\f[R], or has available space less than +\f[C]minfreespace\f[R]. .PP Policies may have their own additional filtering such as those that require existing paths to be present. .PP If all branches are filtered an error will be returned. -Typically \f[B]EROFS\f[] (read\-only filesystem) or \f[B]ENOSPC\f[] (no -space left on device) depending on the most recent reason for filtering -a branch. -\f[B]ENOENT\f[] will be returned if no elegible branch is found. +Typically \f[B]EROFS\f[R] (read\-only filesystem) or \f[B]ENOSPC\f[R] +(no space left on device) depending on the most recent reason for +filtering a branch. +\f[B]ENOENT\f[R] will be returned if no elegible branch is found. .SS Path Preservation .PP Policies, as described below, are of two basic types. -\f[C]path\ preserving\f[] and \f[C]non\-path\ preserving\f[]. +\f[C]path preserving\f[R] and \f[C]non\-path preserving\f[R]. .PP -All policies which start with \f[C]ep\f[] (\f[B]epff\f[], -\f[B]eplfs\f[], \f[B]eplus\f[], \f[B]epmfs\f[], \f[B]eprand\f[]) are -\f[C]path\ preserving\f[]. -\f[C]ep\f[] stands for \f[C]existing\ path\f[]. +All policies which start with \f[C]ep\f[R] (\f[B]epff\f[R], +\f[B]eplfs\f[R], \f[B]eplus\f[R], \f[B]epmfs\f[R], \f[B]eprand\f[R]) are +\f[C]path preserving\f[R]. +\f[C]ep\f[R] stands for \f[C]existing path\f[R]. .PP A path preserving policy will only consider drives where the relative path being accessed already exists. @@ -755,15 +754,15 @@ path being accessed already exists. When using non\-path preserving policies paths will be cloned to target drives as necessary. .PP -With the \f[C]msp\f[] or \f[C]most\ shared\ path\f[] policies they are -defined as \f[C]path\ preserving\f[] for the purpose of controlling -\f[C]link\f[] and \f[C]rename\f[]\[aq]s behaviors since -\f[C]ignorepponrename\f[] is available to disable that behavior. +With the \f[C]msp\f[R] or \f[C]most shared path\f[R] policies they are +defined as \f[C]path preserving\f[R] for the purpose of controlling +\f[C]link\f[R] and \f[C]rename\f[R]\[cq]s behaviors since +\f[C]ignorepponrename\f[R] is available to disable that behavior. In mergerfs v3.0 the path preserving behavior of rename and link will likely be separated from the policy all together. .SS Policy descriptions .PP -A policy\[aq]s behavior differs, as mentioned above, based on the +A policy\[cq]s behavior differs, as mentioned above, based on the function it is used with. Sometimes it really might not make sense to even offer certain policies because they are literally the same as others but it makes things a bit @@ -772,7 +771,7 @@ In mergerfs 3.0 this might change. .PP .TS tab(@); -lw(16.6n) lw(53.4n). +lw(16.2n) lw(53.8n). T{ Policy T}@T{ @@ -782,22 +781,22 @@ _ T{ all T}@T{ -Search: Same as \f[B]epall\f[]. -Action: Same as \f[B]epall\f[]. -Create: for \f[B]mkdir\f[], \f[B]mknod\f[], and \f[B]symlink\f[] it will -apply to all branches. -\f[B]create\f[] works like \f[B]ff\f[]. +Search: Same as \f[B]epall\f[R]. +Action: Same as \f[B]epall\f[R]. +Create: for \f[B]mkdir\f[R], \f[B]mknod\f[R], and \f[B]symlink\f[R] it +will apply to all branches. +\f[B]create\f[R] works like \f[B]ff\f[R]. T} T{ epall (existing path, all) T}@T{ -Search: Same as \f[B]epff\f[] (but more expensive because it doesn\[aq]t -stop after finding a valid branch). +Search: Same as \f[B]epff\f[R] (but more expensive because it +doesn\[cq]t stop after finding a valid branch). Action: apply to all found. -Create: for \f[B]mkdir\f[], \f[B]mknod\f[], and \f[B]symlink\f[] it will -apply to all found. -\f[B]create\f[] works like \f[B]epff\f[] (but more expensive because it -doesn\[aq]t stop after finding a valid branch). +Create: for \f[B]mkdir\f[R], \f[B]mknod\f[R], and \f[B]symlink\f[R] it +will apply to all found. +\f[B]create\f[R] works like \f[B]epff\f[R] (but more expensive because +it doesn\[cq]t stop after finding a valid branch). T} T{ epff (existing path, first found) @@ -826,79 +825,79 @@ T} T{ eppfrd (existing path, percentage free random distribution) T}@T{ -Like \f[B]pfrd\f[] but limited to existing paths. +Like \f[B]pfrd\f[R] but limited to existing paths. T} T{ eprand (existing path, random) T}@T{ -Calls \f[B]epall\f[] and then randomizes. +Calls \f[B]epall\f[R] and then randomizes. Returns 1. T} T{ erofs T}@T{ -Exclusively return \f[B]\-1\f[] with \f[B]errno\f[] set to -\f[B]EROFS\f[] (read\-only filesystem). +Exclusively return \f[B]\-1\f[R] with \f[B]errno\f[R] set to +\f[B]EROFS\f[R] (read\-only filesystem). T} T{ ff (first found) T}@T{ -Search: Same as \f[B]epff\f[]. -Action: Same as \f[B]epff\f[]. +Search: Same as \f[B]epff\f[R]. +Action: Same as \f[B]epff\f[R]. Create: Given the order of the drives, as defined at mount time or configured at runtime, act on the first one found. T} T{ lfs (least free space) T}@T{ -Search: Same as \f[B]eplfs\f[]. -Action: Same as \f[B]eplfs\f[]. +Search: Same as \f[B]eplfs\f[R]. +Action: Same as \f[B]eplfs\f[R]. Create: Pick the drive with the least available free space. T} T{ lus (least used space) T}@T{ -Search: Same as \f[B]eplus\f[]. -Action: Same as \f[B]eplus\f[]. +Search: Same as \f[B]eplus\f[R]. +Action: Same as \f[B]eplus\f[R]. Create: Pick the drive with the least used space. T} T{ mfs (most free space) T}@T{ -Search: Same as \f[B]epmfs\f[]. -Action: Same as \f[B]epmfs\f[]. +Search: Same as \f[B]epmfs\f[R]. +Action: Same as \f[B]epmfs\f[R]. Create: Pick the drive with the most available free space. T} T{ msplfs (most shared path, least free space) T}@T{ -Search: Same as \f[B]eplfs\f[]. -Action: Same as \f[B]eplfs\f[]. -Create: like \f[B]eplfs\f[] but walk back the path if it fails to find a -branch at that level. +Search: Same as \f[B]eplfs\f[R]. +Action: Same as \f[B]eplfs\f[R]. +Create: like \f[B]eplfs\f[R] but walk back the path if it fails to find +a branch at that level. T} T{ msplus (most shared path, least used space) T}@T{ -Search: Same as \f[B]eplus\f[]. -Action: Same as \f[B]eplus\f[]. -Create: like \f[B]eplus\f[] but walk back the path if it fails to find a -branch at that level. +Search: Same as \f[B]eplus\f[R]. +Action: Same as \f[B]eplus\f[R]. +Create: like \f[B]eplus\f[R] but walk back the path if it fails to find +a branch at that level. T} T{ mspmfs (most shared path, most free space) T}@T{ -Search: Same as \f[B]epmfs\f[]. -Action: Same as \f[B]epmfs\f[]. -Create: like \f[B]epmfs\f[] but walk back the path if it fails to find a -branch at that level. +Search: Same as \f[B]epmfs\f[R]. +Action: Same as \f[B]epmfs\f[R]. +Create: like \f[B]epmfs\f[R] but walk back the path if it fails to find +a branch at that level. T} T{ msppfrd (most shared path, percentage free random distribution) T}@T{ -Search: Same as \f[B]eppfrd\f[]. -Action: Same as \f[B]eppfrd\f[]. -Create: Like \f[B]eppfrd\f[] but will walk back the path if it fails to +Search: Same as \f[B]eppfrd\f[R]. +Action: Same as \f[B]eppfrd\f[R]. +Create: Like \f[B]eppfrd\f[R] but will walk back the path if it fails to find a branch at that level. T} T{ @@ -909,27 +908,27 @@ T} T{ pfrd (percentage free random distribution) T}@T{ -Search: Same as \f[B]eppfrd\f[]. -Action: Same as \f[B]eppfrd\f[]. +Search: Same as \f[B]eppfrd\f[R]. +Action: Same as \f[B]eppfrd\f[R]. Create: Chooses a branch at random with the likelihood of selection -based on a branch\[aq]s available space relative to the total. +based on a branch\[cq]s available space relative to the total. T} T{ rand (random) T}@T{ -Calls \f[B]all\f[] and then randomizes. +Calls \f[B]all\f[R] and then randomizes. Returns 1. T} .TE .PP -\f[B]NOTE:\f[] If you are using an underlying filesystem that reserves +\f[B]NOTE:\f[R] If you are using an underlying filesystem that reserves blocks such as ext2, ext3, or ext4 be aware that mergerfs respects the -reservation by using \f[C]f_bavail\f[] (number of free blocks for -unprivileged users) rather than \f[C]f_bfree\f[] (number of free blocks) -in policy calculations. -\f[B]df\f[] does NOT use \f[C]f_bavail\f[], it uses \f[C]f_bfree\f[], so -direct comparisons between \f[B]df\f[] output and mergerfs\[aq] policies -is not appropriate. +reservation by using \f[C]f_bavail\f[R] (number of free blocks for +unprivileged users) rather than \f[C]f_bfree\f[R] (number of free +blocks) in policy calculations. +\f[B]df\f[R] does NOT use \f[C]f_bavail\f[R], it uses \f[C]f_bfree\f[R], +so direct comparisons between \f[B]df\f[R] output and mergerfs\[cq] +policies is not appropriate. .SS Defaults .PP .TS @@ -959,26 +958,27 @@ T} .TE .SS ioctl .PP -When \f[C]ioctl\f[] is used with an open file then it will use the file -handle which was created at the original \f[C]open\f[] call. -However, when using \f[C]ioctl\f[] with a directory mergerfs will use -the \f[C]open\f[] policy to find the directory to act on. +When \f[C]ioctl\f[R] is used with an open file then it will use the file +handle which was created at the original \f[C]open\f[R] call. +However, when using \f[C]ioctl\f[R] with a directory mergerfs will use +the \f[C]open\f[R] policy to find the directory to act on. .SS rename & link .PP -\f[B]NOTE:\f[] If you\[aq]re receiving errors from software when files +\f[B]NOTE:\f[R] If you\[cq]re receiving errors from software when files are moved / renamed / linked then you should consider changing the -create policy to one which is \f[B]not\f[] path preserving, enabling -\f[C]ignorepponrename\f[], or contacting the author of the offending -software and requesting that \f[C]EXDEV\f[] (cross device / improper +create policy to one which is \f[B]not\f[R] path preserving, enabling +\f[C]ignorepponrename\f[R], or contacting the author of the offending +software and requesting that \f[C]EXDEV\f[R] (cross device / improper link) be properly handled. .PP -\f[C]rename\f[] and \f[C]link\f[] are tricky functions in a union +\f[C]rename\f[R] and \f[C]link\f[R] are tricky functions in a union filesystem. -\f[C]rename\f[] only works within a single filesystem or device. -If a rename can\[aq]t be done atomically due to the source and +\f[C]rename\f[R] only works within a single filesystem or device. +If a rename can\[cq]t be done atomically due to the source and destination paths existing on different mount points it will return -\f[B]\-1\f[] with \f[B]errno = EXDEV\f[] (cross device / improper link). -So if a \f[C]rename\f[]\[aq]s source and target are on different drives +\f[B]\-1\f[R] with \f[B]errno = EXDEV\f[R] (cross device / improper +link). +So if a \f[C]rename\f[R]\[cq]s source and target are on different drives within the pool it creates an issue. .PP Originally mergerfs would return EXDEV whenever a rename was requested @@ -988,21 +988,22 @@ requirements. However, many applications fail to handle EXDEV at all and treat it as a normal error or otherwise handle it poorly. Such apps include: gvfsd\-fuse v1.20.3 and prior, Finder / CIFS/SMB -client in Apple OSX 10.9+, NZBGet, Samba\[aq]s recycling bin feature. +client in Apple OSX 10.9+, NZBGet, Samba\[cq]s recycling bin feature. .PP As a result a compromise was made in order to get most software to work -while still obeying mergerfs\[aq] policies. +while still obeying mergerfs\[cq] policies. Below is the basic logic. .IP \[bu] 2 -If using a \f[B]create\f[] policy which tries to preserve directory +If using a \f[B]create\f[R] policy which tries to preserve directory paths (epff,eplfs,eplus,epmfs) +.RS 2 .IP \[bu] 2 -Using the \f[B]rename\f[] policy get the list of files to rename +Using the \f[B]rename\f[R] policy get the list of files to rename .IP \[bu] 2 For each file attempt rename: .RS 2 .IP \[bu] 2 -If failure with ENOENT (no such file or directory) run \f[B]create\f[] +If failure with ENOENT (no such file or directory) run \f[B]create\f[R] policy .IP \[bu] 2 If create policy returns the same drive as currently evaluating then @@ -1011,10 +1012,10 @@ clone the path Re\-attempt rename .RE .IP \[bu] 2 -If \f[B]any\f[] of the renames succeed the higher level rename is +If \f[B]any\f[R] of the renames succeed the higher level rename is considered a success .IP \[bu] 2 -If \f[B]no\f[] renames succeed the first error encountered will be +If \f[B]no\f[R] renames succeed the first error encountered will be returned .IP \[bu] 2 On success: @@ -1024,28 +1025,32 @@ Remove the target from all drives with no source file .IP \[bu] 2 Remove the source from all drives which failed to rename .RE +.RE .IP \[bu] 2 -If using a \f[B]create\f[] policy which does \f[B]not\f[] try to +If using a \f[B]create\f[R] policy which does \f[B]not\f[R] try to preserve directory paths +.RS 2 .IP \[bu] 2 -Using the \f[B]rename\f[] policy get the list of files to rename +Using the \f[B]rename\f[R] policy get the list of files to rename .IP \[bu] 2 -Using the \f[B]getattr\f[] policy get the target path +Using the \f[B]getattr\f[R] policy get the target path .IP \[bu] 2 For each file attempt rename: .RS 2 .IP \[bu] 2 If the source drive != target drive: +.RS 2 .IP \[bu] 2 Clone target path from target drive to source drive +.RE .IP \[bu] 2 Rename .RE .IP \[bu] 2 -If \f[B]any\f[] of the renames succeed the higher level rename is +If \f[B]any\f[R] of the renames succeed the higher level rename is considered a success .IP \[bu] 2 -If \f[B]no\f[] renames succeed the first error encountered will be +If \f[B]no\f[R] renames succeed the first error encountered will be returned .IP \[bu] 2 On success: @@ -1055,13 +1060,14 @@ Remove the target from all drives with no source file .IP \[bu] 2 Remove the source from all drives which failed to rename .RE +.RE .PP The the removals are subject to normal entitlement checks. .PP The above behavior will help minimize the likelihood of EXDEV being returned but it will still be possible. .PP -\f[B]link\f[] uses the same strategy but without the removals. +\f[B]link\f[R] uses the same strategy but without the removals. .SS readdir .PP readdir (http://linux.die.net/man/3/readdir) is different from all other @@ -1070,11 +1076,11 @@ While it could have its own set of policies to tweak its behavior at this time it provides a simple union of files and directories found. Remember that any action or information queried about these files and directories come from the respective function. -For instance: an \f[B]ls\f[] is a \f[B]readdir\f[] and for each -file/directory returned \f[B]getattr\f[] is called. -Meaning the policy of \f[B]getattr\f[] is responsible for choosing the +For instance: an \f[B]ls\f[R] is a \f[B]readdir\f[R] and for each +file/directory returned \f[B]getattr\f[R] is called. +Meaning the policy of \f[B]getattr\f[R] is responsible for choosing the file/directory which is the source of the metadata you see in an -\f[B]ls\f[]. +\f[B]ls\f[R]. .SS statfs / statvfs .PP statvfs (http://linux.die.net/man/2/statvfs) normalizes the source @@ -1085,10 +1091,10 @@ Total, used, and free. The sources however are dedupped based on the drive so multiple sources on the same drive will not result in double counting its space. Filesystems mounted further down the tree of the branch will not be -included when checking the mount\[aq]s stats. +included when checking the mount\[cq]s stats. .PP -The options \f[C]statfs\f[] and \f[C]statfs_ignore\f[] can be used to -modify \f[C]statfs\f[] behavior. +The options \f[C]statfs\f[R] and \f[C]statfs_ignore\f[R] can be used to +modify \f[C]statfs\f[R] behavior. .SH ERROR HANDLING .PP POSIX filesystem functions offer a single return code meaning that there @@ -1125,47 +1131,47 @@ For search functions there is always a single thing acted on and as such whatever return value that comes from the single function call is returned. .PP -For create functions \f[C]mkdir\f[], \f[C]mknod\f[], and -\f[C]symlink\f[] which don\[aq]t return a file descriptor and therefore -can have \f[C]all\f[] or \f[C]epall\f[] policies it will return success -if any of the calls succeed and an error otherwise. +For create functions \f[C]mkdir\f[R], \f[C]mknod\f[R], and +\f[C]symlink\f[R] which don\[cq]t return a file descriptor and therefore +can have \f[C]all\f[R] or \f[C]epall\f[R] policies it will return +success if any of the calls succeed and an error otherwise. .SH BUILD / UPDATE .PP -\f[B]NOTE:\f[] Prebuilt packages can be found at and recommended for -most users: https://github.com/trapexit/mergerfs/releases \f[B]NOTE:\f[] -Only tagged releases are supported. -\f[C]master\f[] and other branches should be considered works in +\f[B]NOTE:\f[R] Prebuilt packages can be found at and recommended for +most users: https://github.com/trapexit/mergerfs/releases +\f[B]NOTE:\f[R] Only tagged releases are supported. +\f[C]master\f[R] and other branches should be considered works in progress. .PP First get the code from github (https://github.com/trapexit/mergerfs). .IP .nf \f[C] -$\ git\ clone\ https://github.com/trapexit/mergerfs.git -$\ #\ or -$\ wget\ https://github.com/trapexit/mergerfs/releases/download//mergerfs\-.tar.gz -\f[] +$ git clone https://github.com/trapexit/mergerfs.git +$ # or +$ wget https://github.com/trapexit/mergerfs/releases/download//mergerfs\-.tar.gz +\f[R] .fi .SS Debian / Ubuntu .IP .nf \f[C] -$\ cd\ mergerfs -$\ sudo\ tools/install\-build\-pkgs -$\ make\ deb -$\ sudo\ dpkg\ \-i\ ../mergerfs__.deb -\f[] +$ cd mergerfs +$ sudo tools/install\-build\-pkgs +$ make deb +$ sudo dpkg \-i ../mergerfs__.deb +\f[R] .fi .SS RHEL / CentOS /Fedora .IP .nf \f[C] -$\ su\ \- -#\ cd\ mergerfs -#\ tools/install\-build\-pkgs -#\ make\ rpm -#\ rpm\ \-i\ rpmbuild/RPMS//mergerfs\-..rpm -\f[] +$ su \- +# cd mergerfs +# tools/install\-build\-pkgs +# make rpm +# rpm \-i rpmbuild/RPMS//mergerfs\-..rpm +\f[R] .fi .SS Generically .PP @@ -1173,22 +1179,22 @@ Have git, g++, make, python installed. .IP .nf \f[C] -$\ cd\ mergerfs -$\ make -$\ sudo\ make\ install -\f[] +$ cd mergerfs +$ make +$ sudo make install +\f[R] .fi .SS Build options .IP .nf \f[C] -$\ make\ help -usage:\ make +$ make help +usage: make -make\ USE_XATTR=0\ \ \ \ \ \ \-\ build\ program\ without\ xattrs\ functionality -make\ STATIC=1\ \ \ \ \ \ \ \ \ \-\ build\ static\ binary -make\ LTO=1\ \ \ \ \ \ \ \ \ \ \ \ \-\ build\ with\ link\ time\ optimization -\f[] +make USE_XATTR=0 \- build program without xattrs functionality +make STATIC=1 \- build static binary +make LTO=1 \- build with link time optimization +\f[R] .fi .SH UPGRADE .PP @@ -1197,28 +1203,28 @@ instance. Simply install the new version of mergerfs and follow the instructions below. .PP -Add \f[C]nonempty\f[] to your mergerfs option list and call mergerfs -again or if using \f[C]/etc/fstab\f[] call for it to mount again. +Add \f[C]nonempty\f[R] to your mergerfs option list and call mergerfs +again or if using \f[C]/etc/fstab\f[R] call for it to mount again. Existing open files and such will continue to work fine though they -won\[aq]t see runtime changes since any such change would be the new +won\[cq]t see runtime changes since any such change would be the new mount. If you plan on changing settings with the new mount you should / could apply those before mounting the new version. .IP .nf \f[C] -$\ sudo\ mount\ /mnt/mergerfs -$\ mount\ |\ grep\ mergerfs -media\ on\ /mnt/mergerfs\ type\ fuse.mergerfs\ (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) -media\ on\ /mnt/mergerfs\ type\ fuse.mergerfs\ (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) -\f[] +$ sudo mount /mnt/mergerfs +$ mount | grep mergerfs +media on /mnt/mergerfs type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) +media on /mnt/mergerfs type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) +\f[R] .fi .PP A problem with this approach is that the underlying instance will continue to run even if the software using it stop or are restarted. -To work around this you can use a "lazy umount". +To work around this you can use a \[lq]lazy umount\[rq]. Before mounting over top the mount point with the new instance of -mergerfs issue: \f[C]umount\ \-l\ \f[]. +mergerfs issue: \f[C]umount \-l \f[R]. .SH RUNTIME CONFIG .SS ioctl .PP @@ -1233,35 +1239,42 @@ the config file. .PP All commands take a 4096 byte char buffer. .IP \[bu] 2 -read keys: get a nul \[aq]\[aq] delimited list of option keys +read keys: get a nul `\[rs]0' delimited list of option keys +.RS 2 .IP \[bu] 2 _IOWR(0xDF,0,char[4096]) = 0xD000DF00 .IP \[bu] 2 on success ioctl return value is the total length +.RE .IP \[bu] 2 read value: get an option value +.RS 2 .IP \[bu] 2 _IOWR(0xDF,1,char[4096]) = 0xD000DF01 .IP \[bu] 2 -the key is passed in via the char buffer as a nul \[aq]\[aq] terminated +the key is passed in via the char buffer as a nul `\[rs]0' terminated string .IP \[bu] 2 on success ioctl return value is the total length +.RE .IP \[bu] 2 write value: set an option value +.RS 2 .IP \[bu] 2 _IOW(0xDF,2,char[4096]) = 0x5000DF02 .IP \[bu] 2 -the key and value is passed in via the char buffer as a nul \[aq]\[aq] -terminated string in the format of \f[C]key=value\f[] +the key and value is passed in via the char buffer as a nul `\[rs]0' +terminated string in the format of \f[C]key=value\f[R] .IP \[bu] 2 on success ioctl return value is 0 +.RE .IP \[bu] 2 file info: get mergerfs metadata info for a file +.RS 2 .IP \[bu] 2 _IOWR(0xDF,3,char[4096]) = 0xD000DF03 .IP \[bu] 2 -the key is passed in via the char buffer as a nul \[aq]\[aq] terminated +the key is passed in via the char buffer as a nul `\[rs]0' terminated string .IP \[bu] 2 on success the ioctl return value is the total length @@ -1277,8 +1290,8 @@ relpath: the relative path of the file from the mount point fullpath: the full path of the underlying file according to the getattr policy .IP \[bu] 2 -allpaths: a NUL \[aq]\[aq] delimited list of full paths to all files -found +allpaths: a NUL `\[rs]0' delimited list of full paths to all files found +.RE .RE .SS .mergerfs pseudo file (deprecated) .PP @@ -1287,32 +1300,31 @@ NOTE: this interface will be removed in mergerfs 3.0 .nf \f[C] /.mergerfs -\f[] +\f[R] .fi .PP There is a pseudo file available at the mount point which allows for the -runtime modification of certain \f[B]mergerfs\f[] options. -The file will not show up in \f[B]readdir\f[] but can be -\f[B]stat\f[]\[aq]ed and manipulated via +runtime modification of certain \f[B]mergerfs\f[R] options. +The file will not show up in \f[B]readdir\f[R] but can be +\f[B]stat\f[R]\[cq]ed and manipulated via {list,get,set}xattrs (http://linux.die.net/man/2/listxattr) calls. .PP -Any changes made at runtime are \f[B]not\f[] persisted. +Any changes made at runtime are \f[B]not\f[R] persisted. If you wish for values to persist they must be included as options wherever you configure the mounting of mergerfs (/etc/fstab). .SS Keys .PP -Use \f[C]xattr\ \-l\ /mountpoint/.mergerfs\f[] to see all supported -keys. +Use \f[C]xattr \-l /mountpoint/.mergerfs\f[R] to see all supported keys. Some are informational and therefore read\-only. -\f[C]setxattr\f[] will return EINVAL (invalid argument) on read\-only +\f[C]setxattr\f[R] will return EINVAL (invalid argument) on read\-only keys. .SS Values .PP Same as the command line. .SS user.mergerfs.branches .PP -\f[B]NOTE:\f[] formerly \f[C]user.mergerfs.srcmounts\f[] but said key is -still supported. +\f[B]NOTE:\f[R] formerly \f[C]user.mergerfs.srcmounts\f[R] but said key +is still supported. .PP Used to query or modify the list of branches. When modifying there are several shortcuts to easy manipulation of the @@ -1359,79 +1371,80 @@ remove last in list T} .TE .PP -\f[C]xattr\ \-w\ user.mergerfs.branches\ + userspace round trips there are kernel side caches for file entries and attributes. -The entry cache limits the \f[C]lookup\f[] calls to mergerfs which ask +The entry cache limits the \f[C]lookup\f[R] calls to mergerfs which ask if a file exists. -The attribute cache limits the need to make \f[C]getattr\f[] calls to +The attribute cache limits the need to make \f[C]getattr\f[R] calls to mergerfs which provide file attributes (mode, size, type, etc.). As with the page cache these should not be used if the underlying filesystems are being manipulated at the same time as it could lead to odd behavior or data corruption. -The options for setting these are \f[C]cache.entry\f[] and -\f[C]cache.negative_entry\f[] for the entry cache and -\f[C]cache.attr\f[] for the attributes cache. -\f[C]cache.negative_entry\f[] refers to the timeout for negative +The options for setting these are \f[C]cache.entry\f[R] and +\f[C]cache.negative_entry\f[R] for the entry cache and +\f[C]cache.attr\f[R] for the attributes cache. +\f[C]cache.negative_entry\f[R] refers to the timeout for negative responses to lookups (non\-existent files). .SS writeback caching .PP -When \f[C]cache.files\f[] is enabled the default is for it to perform +When \f[C]cache.files\f[R] is enabled the default is for it to perform writethrough caching. -This behavior won\[aq]t help improve performance as each write still +This behavior won\[cq]t help improve performance as each write still goes one for one through the filesystem. By enabling the FUSE writeback cache small writes may be aggregated by the kernel and then sent to mergerfs as one larger request. @@ -1549,15 +1567,15 @@ This can greatly improve the throughput for apps which write to files inefficiently. The amount the kernel can aggregate is limited by the size of a FUSE message. -Read the \f[C]fuse_msg_size\f[] section for more details. +Read the \f[C]fuse_msg_size\f[R] section for more details. .PP There is a small side effect as a result of enabling writeback caching. -Underlying files won\[aq]t ever be opened with O_APPEND or O_WRONLY. +Underlying files won\[cq]t ever be opened with O_APPEND or O_WRONLY. The former because the kernel then manages append mode and the latter because the kernel may request file data from mergerfs to populate the write cache. The O_APPEND change means that if a file is changed outside of mergerfs -it could lead to corruption as the kernel won\[aq]t know the end of the +it could lead to corruption as the kernel won\[cq]t know the end of the file has changed. That said any time you use caching you should keep from using the same file outside of mergerfs at the same time. @@ -1570,62 +1588,64 @@ on older kernels, 1M on newer). .PP Policies are run every time a function (with a policy as mentioned above) is called. -These policies can be expensive depending on mergerfs\[aq] setup and +These policies can be expensive depending on mergerfs\[cq] setup and client usage patterns. -Generally we wouldn\[aq]t want to cache policy results because it may +Generally we wouldn\[cq]t want to cache policy results because it may result in stale responses if the underlying drives are used directly. .PP -The \f[C]open\f[] policy cache will cache the result of an \f[C]open\f[] -policy for a particular input for \f[C]cache.open\f[] seconds or until -the file is unlinked. +The \f[C]open\f[R] policy cache will cache the result of an +\f[C]open\f[R] policy for a particular input for \f[C]cache.open\f[R] +seconds or until the file is unlinked. Each file close (release) will randomly chose to clean up the cache of expired entries. .PP This cache is really only useful in cases where you have a large number -of branches and \f[C]open\f[] is called on the same files repeatedly -(like \f[B]Transmission\f[] which opens and closes a file on every +of branches and \f[C]open\f[R] is called on the same files repeatedly +(like \f[B]Transmission\f[R] which opens and closes a file on every read/write presumably to keep file handle usage low). .SS statfs caching .PP -Of the syscalls used by mergerfs in policies the \f[C]statfs\f[] / -\f[C]statvfs\f[] call is perhaps the most expensive. -It\[aq]s used to find out the available space of a drive and whether it +Of the syscalls used by mergerfs in policies the \f[C]statfs\f[R] / +\f[C]statvfs\f[R] call is perhaps the most expensive. +It\[cq]s used to find out the available space of a drive and whether it is mounted read\-only. Depending on the setup and usage pattern these queries can be relatively costly. -When \f[C]cache.statfs\f[] is enabled all calls to \f[C]statfs\f[] by a -policy will be cached for the number of seconds its set to. +When \f[C]cache.statfs\f[R] is enabled all calls to \f[C]statfs\f[R] by +a policy will be cached for the number of seconds its set to. .PP -Example: If the create policy is \f[C]mfs\f[] and the timeout is 60 then -for that 60 seconds the same drive will be returned as the target for -creates because the available space won\[aq]t be updated for that time. +Example: If the create policy is \f[C]mfs\f[R] and the timeout is 60 +then for that 60 seconds the same drive will be returned as the target +for creates because the available space won\[cq]t be updated for that +time. .SS symlink caching .PP As of version 4.20 Linux supports symlink caching. Significant performance increases can be had in workloads which use a lot of symlinks. -Setting \f[C]cache.symlinks=true\f[] will result in requesting symlink +Setting \f[C]cache.symlinks=true\f[R] will result in requesting symlink caching from the kernel only if supported. As a result its safe to enable it on systems prior to 4.20. That said it is disabled by default for now. You can see if caching is enabled by querying the xattr -\f[C]user.mergerfs.cache.symlinks\f[] but given it must be requested at +\f[C]user.mergerfs.cache.symlinks\f[R] but given it must be requested at startup you can not change it at runtime. .SS readdir caching .PP As of version 4.20 Linux supports readdir caching. This can have a significant impact on directory traversal. -Especially when combined with entry (\f[C]cache.entry\f[]) and attribute -(\f[C]cache.attr\f[]) caching. -Setting \f[C]cache.readdir=true\f[] will result in requesting readdir -caching from the kernel on each \f[C]opendir\f[]. -If the kernel doesn\[aq]t support readdir caching setting the option to -\f[C]true\f[] has no effect. +Especially when combined with entry (\f[C]cache.entry\f[R]) and +attribute (\f[C]cache.attr\f[R]) caching. +Setting \f[C]cache.readdir=true\f[R] will result in requesting readdir +caching from the kernel on each \f[C]opendir\f[R]. +If the kernel doesn\[cq]t support readdir caching setting the option to +\f[C]true\f[R] has no effect. This option is configurable at runtime via xattr -\f[C]user.mergerfs.cache.readdir\f[]. +\f[C]user.mergerfs.cache.readdir\f[R]. .SS tiered caching .PP -Some storage technologies support what some call "tiered" caching. +Some storage technologies support what some call \[lq]tiered\[rq] +caching. The placing of usually smaller, faster storage as a transparent cache to larger, slower storage. NVMe, SSD, Optane in front of traditional HDDs for instance. @@ -1636,18 +1656,18 @@ complicate the code. However, there are a few situations where a cache drive could help with a typical mergerfs setup. .IP "1." 3 -Fast network, slow drives, many readers: You\[aq]ve a 10+Gbps network -with many readers and your regular drives can\[aq]t keep up. +Fast network, slow drives, many readers: You\[cq]ve a 10+Gbps network +with many readers and your regular drives can\[cq]t keep up. .IP "2." 3 -Fast network, slow drives, small\[aq]ish bursty writes: You have a +Fast network, slow drives, small\[cq]ish bursty writes: You have a 10+Gbps network and wish to transfer amounts of data less than your cache drive but wish to do so quickly. .PP With #1 its arguable if you should be using mergerfs at all. RAID would probably be the better solution. -If you\[aq]re going to use mergerfs there are other tactics that may +If you\[cq]re going to use mergerfs there are other tactics that may help: spreading the data across drives (see the mergerfs.dup tool) and -setting \f[C]func.open=rand\f[], using \f[C]symlinkify\f[], or using +setting \f[C]func.open=rand\f[R], using \f[C]symlinkify\f[R], or using dm\-cache or a similar technology to add tiered cache to the underlying device. .PP @@ -1658,90 +1678,101 @@ Create 2 mergerfs pools. One which includes just the slow drives and one which has both the fast drives (SSD,NVME,etc.) and slow drives. .IP "2." 3 -The \[aq]cache\[aq] pool should have the cache drives listed first. +The `cache' pool should have the cache drives listed first. .IP "3." 3 -The best \f[C]create\f[] policies to use for the \[aq]cache\[aq] pool -would probably be \f[C]ff\f[], \f[C]epff\f[], \f[C]lfs\f[], or -\f[C]eplfs\f[]. +The best \f[C]create\f[R] policies to use for the `cache' pool would +probably be \f[C]ff\f[R], \f[C]epff\f[R], \f[C]lfs\f[R], or +\f[C]eplfs\f[R]. The latter two under the assumption that the cache drive(s) are far smaller than the backing drives. -If using path preserving policies remember that you\[aq]ll need to +If using path preserving policies remember that you\[cq]ll need to manually create the core directories of those paths you wish to be cached. Be sure the permissions are in sync. -Use \f[C]mergerfs.fsck\f[] to check / correct them. -You could also tag the slow drives as \f[C]=NC\f[] though that\[aq]d -mean if the cache drives fill you\[aq]d get "out of space" errors. +Use \f[C]mergerfs.fsck\f[R] to check / correct them. +You could also tag the slow drives as \f[C]=NC\f[R] though that\[cq]d +mean if the cache drives fill you\[cq]d get \[lq]out of space\[rq] +errors. .IP "4." 3 -Enable \f[C]moveonenospc\f[] and set \f[C]minfreespace\f[] +Enable \f[C]moveonenospc\f[R] and set \f[C]minfreespace\f[R] appropriately. -To make sure there is enough room on the "slow" pool you might want to -set \f[C]minfreespace\f[] to at least as large as the size of the -largest cache drive if not larger. +To make sure there is enough room on the \[lq]slow\[rq] pool you might +want to set \f[C]minfreespace\f[R] to at least as large as the size of +the largest cache drive if not larger. This way in the worst case the whole of the cache drive(s) can be moved to the other drives. .IP "5." 3 Set your programs to use the cache pool. .IP "6." 3 -Save one of the below scripts or create you\[aq]re own. +Save one of the below scripts or create you\[cq]re own. .IP "7." 3 -Use \f[C]cron\f[] (as root) to schedule the command at whatever +Use \f[C]cron\f[R] (as root) to schedule the command at whatever frequency is appropriate for your workflow. .SS time based expiring .PP Move files from cache to backing pool based only on the last time the file was accessed. -Replace \f[C]\-atime\f[] with \f[C]\-amin\f[] if you want minutes rather -than days. -May want to use the \f[C]fadvise\f[] / \f[C]\-\-drop\-cache\f[] version -of rsync or run rsync with the tool "nocache". +Replace \f[C]\-atime\f[R] with \f[C]\-amin\f[R] if you want minutes +rather than days. +May want to use the \f[C]fadvise\f[R] / \f[C]\-\-drop\-cache\f[R] +version of rsync or run rsync with the tool \[lq]nocache\[rq]. +.PP +\f[I]NOTE:\f[R] The arguments to these scripts include the cache +\f[B]drive\f[R]. +Not the pool with the cache drive. +You could have data loss if the source is the cache pool. .IP .nf \f[C] #!/bin/bash -if\ [\ $#\ !=\ 3\ ];\ then -\ \ echo\ "usage:\ $0\ \ \ " -\ \ exit\ 1 +if [ $# != 3 ]; then + echo \[dq]usage: $0 \[dq] + exit 1 fi -CACHE="${1}" -BACKING="${2}" +CACHE=\[dq]${1}\[dq] +BACKING=\[dq]${2}\[dq] N=${3} -find\ "${CACHE}"\ \-type\ f\ \-atime\ +${N}\ \-printf\ \[aq]%P\\n\[aq]\ |\ \\ -\ \ rsync\ \-\-files\-from=\-\ \-axqHAXWES\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/"\ "${BACKING}/" -\f[] +find \[dq]${CACHE}\[dq] \-type f \-atime +${N} \-printf \[aq]%P\[rs]n\[aq] | \[rs] + rsync \-\-files\-from=\- \-axqHAXWES \-\-preallocate \-\-remove\-source\-files \[dq]${CACHE}/\[dq] \[dq]${BACKING}/\[dq] +\f[R] .fi .SS percentage full expiring .PP Move the oldest file from the cache to the backing pool. Continue till below percentage threshold. +.PP +\f[I]NOTE:\f[R] The arguments to these scripts include the cache +\f[B]drive\f[R]. +Not the pool with the cache drive. +You could have data loss if the source is the cache pool. .IP .nf \f[C] #!/bin/bash -if\ [\ $#\ !=\ 3\ ];\ then -\ \ echo\ "usage:\ $0\ \ \ " -\ \ exit\ 1 +if [ $# != 3 ]; then + echo \[dq]usage: $0 \[dq] + exit 1 fi -CACHE="${1}" -BACKING="${2}" +CACHE=\[dq]${1}\[dq] +BACKING=\[dq]${2}\[dq] PERCENTAGE=${3} -set\ \-o\ errexit -while\ [\ $(df\ \-\-output=pcent\ "${CACHE}"\ |\ grep\ \-v\ Use\ |\ cut\ \-d\[aq]%\[aq]\ \-f1)\ \-gt\ ${PERCENTAGE}\ ] +set \-o errexit +while [ $(df \-\-output=pcent \[dq]${CACHE}\[dq] | grep \-v Use | cut \-d\[aq]%\[aq] \-f1) \-gt ${PERCENTAGE} ] do -\ \ \ \ FILE=$(find\ "${CACHE}"\ \-type\ f\ \-printf\ \[aq]%A\@\ %P\\n\[aq]\ |\ \\ -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ sort\ |\ \\ -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ head\ \-n\ 1\ |\ \\ -\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ cut\ \-d\[aq]\ \[aq]\ \-f2\-) -\ \ \ \ test\ \-n\ "${FILE}" -\ \ \ \ rsync\ \-axqHAXWESR\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/./${FILE}"\ "${BACKING}/" + FILE=$(find \[dq]${CACHE}\[dq] \-type f \-printf \[aq]%A\[at] %P\[rs]n\[aq] | \[rs] + sort | \[rs] + head \-n 1 | \[rs] + cut \-d\[aq] \[aq] \-f2\-) + test \-n \[dq]${FILE}\[dq] + rsync \-axqHAXWESR \-\-preallocate \-\-remove\-source\-files \[dq]${CACHE}/./${FILE}\[dq] \[dq]${BACKING}/\[dq] done -\f[] +\f[R] .fi .SH PERFORMANCE .PP @@ -1750,49 +1781,49 @@ performance is that of the underlying devices. However, given it is a FUSE filesystem working from userspace there is an increase in overhead relative to kernel based solutions. That said the performance can match the theoretical max but it depends -greatly on the system\[aq]s configuration. +greatly on the system\[cq]s configuration. Especially when adding network filesystems into the mix there are many variables which can impact performance. Drive speeds and latency, network speeds and latency, general concurrency, read/write sizes, etc. Unfortunately, given the number of variables it has been difficult to find a single set of settings which provide optimal performance. -If you\[aq]re having performance issues please look over the suggestions +If you\[cq]re having performance issues please look over the suggestions below (including the benchmarking section.) .PP NOTE: be sure to read about these features before changing them to understand what behaviors it may impact .IP \[bu] 2 -enable (or disable) \f[C]splice_move\f[], \f[C]splice_read\f[], and -\f[C]splice_write\f[] +enable (or disable) \f[C]splice_move\f[R], \f[C]splice_read\f[R], and +\f[C]splice_write\f[R] .IP \[bu] 2 -disable \f[C]security_capability\f[] and/or \f[C]xattr\f[] +disable \f[C]security_capability\f[R] and/or \f[C]xattr\f[R] .IP \[bu] 2 -increase cache timeouts \f[C]cache.attr\f[], \f[C]cache.entry\f[], -\f[C]cache.negative_entry\f[] +increase cache timeouts \f[C]cache.attr\f[R], \f[C]cache.entry\f[R], +\f[C]cache.negative_entry\f[R] .IP \[bu] 2 -enable (or disable) page caching (\f[C]cache.files\f[]) +enable (or disable) page caching (\f[C]cache.files\f[R]) .IP \[bu] 2 -enable \f[C]cache.writeback\f[] +enable \f[C]cache.writeback\f[R] .IP \[bu] 2 -enable \f[C]cache.open\f[] +enable \f[C]cache.open\f[R] .IP \[bu] 2 -enable \f[C]cache.statfs\f[] +enable \f[C]cache.statfs\f[R] .IP \[bu] 2 -enable \f[C]cache.symlinks\f[] +enable \f[C]cache.symlinks\f[R] .IP \[bu] 2 -enable \f[C]cache.readdir\f[] +enable \f[C]cache.readdir\f[R] .IP \[bu] 2 change the number of worker threads .IP \[bu] 2 -disable \f[C]posix_acl\f[] +disable \f[C]posix_acl\f[R] .IP \[bu] 2 -disable \f[C]async_read\f[] +disable \f[C]async_read\f[R] .IP \[bu] 2 -test theoretical performance using \f[C]nullrw\f[] or mounting a ram +test theoretical performance using \f[C]nullrw\f[R] or mounting a ram disk .IP \[bu] 2 -use \f[C]symlinkify\f[] if your data is largely static and read\-only +use \f[C]symlinkify\f[R] if your data is largely static and read\-only .IP \[bu] 2 use tiered cache drives .IP \[bu] 2 @@ -1806,40 +1837,40 @@ Filesystems are complicated. They do many things and many of those are interconnected. Additionally, the OS, drivers, hardware, etc. all can impact performance. -Therefore, when benchmarking, it is \f[B]necessary\f[] that the test +Therefore, when benchmarking, it is \f[B]necessary\f[R] that the test focus as narrowly as possible. .PP For most throughput is the key benchmark. -To test throughput \f[C]dd\f[] is useful but \f[B]must\f[] be used with -the correct settings in order to ensure the filesystem or device is +To test throughput \f[C]dd\f[R] is useful but \f[B]must\f[R] be used +with the correct settings in order to ensure the filesystem or device is actually being tested. The OS can and will cache data. Without forcing synchronous reads and writes and/or disabling caching -the values returned will not be representative of the device\[aq]s true +the values returned will not be representative of the device\[cq]s true performance. .PP When benchmarking through mergerfs ensure you only use 1 branch to remove any possibility of the policies complicating the situation. Benchmark the underlying filesystem first and then mount mergerfs over it and test again. -If you\[aq]re experience speeds below your expectation you will need to +If you\[cq]re experience speeds below your expectation you will need to narrow down precisely which component is leading to the slowdown. Preferably test the following in the order listed (but not combined). .IP "1." 3 -Enable \f[C]nullrw\f[] mode with \f[C]nullrw=true\f[]. +Enable \f[C]nullrw\f[R] mode with \f[C]nullrw=true\f[R]. This will effectively make reads and writes no\-ops. Removing the underlying device / filesystem from the equation. This will give us the top theoretical speeds. .IP "2." 3 -Mount mergerfs over \f[C]tmpfs\f[]. -\f[C]tmpfs\f[] is a RAM disk. +Mount mergerfs over \f[C]tmpfs\f[R]. +\f[C]tmpfs\f[R] is a RAM disk. Extremely high speed and very low latency. This is a more realistic best case scenario. -Example: \f[C]mount\ \-t\ tmpfs\ \-o\ size=2G\ tmpfs\ /tmp/tmpfs\f[] +Example: \f[C]mount \-t tmpfs \-o size=2G tmpfs /tmp/tmpfs\f[R] .IP "3." 3 Mount mergerfs over a local drive. NVMe, SSD, HDD, etc. -If you have more than one I\[aq]d suggest testing each of them as drives +If you have more than one I\[cq]d suggest testing each of them as drives and/or controllers (their drivers) could impact performance. .IP "4." 3 Finally, if you intend to use mergerfs with a network filesystem, either @@ -1849,12 +1880,12 @@ each of those alone as above. Once you find the component which has the performance issue you can do further testing with different options to see if they impact performance. -For reads and writes the most relevant would be: \f[C]cache.files\f[], -\f[C]async_read\f[], \f[C]splice_move\f[], \f[C]splice_read\f[], -\f[C]splice_write\f[]. +For reads and writes the most relevant would be: \f[C]cache.files\f[R], +\f[C]async_read\f[R], \f[C]splice_move\f[R], \f[C]splice_read\f[R], +\f[C]splice_write\f[R]. Less likely but relevant when using NFS or with certain filesystems -would be \f[C]security_capability\f[], \f[C]xattr\f[], and -\f[C]posix_acl\f[]. +would be \f[C]security_capability\f[R], \f[C]xattr\f[R], and +\f[C]posix_acl\f[R]. If you find a specific system, drive, filesystem, controller, etc. that performs poorly contact trapexit so he may investigate further. .PP @@ -1862,11 +1893,11 @@ Sometimes the problem is really the application accessing or writing data through mergerfs. Some software use small buffer sizes which can lead to more requests and therefore greater overhead. -You can test this out yourself by replace \f[C]bs=1M\f[] in the examples -below with \f[C]ibs\f[] or \f[C]obs\f[] and using a size of \f[C]512\f[] -instead of \f[C]1M\f[]. -In one example test using \f[C]nullrw\f[] the write speed dropped from -4.9GB/s to 69.7MB/s when moving from \f[C]1M\f[] to \f[C]512\f[]. +You can test this out yourself by replace \f[C]bs=1M\f[R] in the +examples below with \f[C]ibs\f[R] or \f[C]obs\f[R] and using a size of +\f[C]512\f[R] instead of \f[C]1M\f[R]. +In one example test using \f[C]nullrw\f[R] the write speed dropped from +4.9GB/s to 69.7MB/s when moving from \f[C]1M\f[R] to \f[C]512\f[R]. Similar results were had when testing reads. Small writes overhead may be improved by leveraging a write cache but in casual tests little gain was found. @@ -1879,72 +1910,73 @@ Contact trapexit so he may investigate further. .IP .nf \f[C] -$\ dd\ if=/dev/zero\ of=/mnt/mergerfs/1GB.file\ bs=1M\ count=1024\ oflag=nocache\ conv=fdatasync\ status=progress -\f[] +$ dd if=/dev/zero of=/mnt/mergerfs/1GB.file bs=1M count=1024 oflag=nocache conv=fdatasync status=progress +\f[R] .fi .SS read benchmark .IP .nf \f[C] -$\ dd\ if=/mnt/mergerfs/1GB.file\ of=/dev/null\ bs=1M\ count=1024\ iflag=nocache\ conv=fdatasync\ status=progress -\f[] +$ dd if=/mnt/mergerfs/1GB.file of=/dev/null bs=1M count=1024 iflag=nocache conv=fdatasync status=progress +\f[R] .fi .SH TIPS / NOTES .IP \[bu] 2 This document is very literal and thorough. Unless there is a bug things work as described. -If a suspected feature isn\[aq]t mentioned it doesn\[aq]t exist. +If a suspected feature isn\[cq]t mentioned it doesn\[cq]t exist. .IP \[bu] 2 -Ensure you\[aq]re using the latest version. +Ensure you\[cq]re using the latest version. Few distros have the latest version. .IP \[bu] 2 -\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and +\f[B]use_ino\f[R] will only work when used with mergerfs 2.18.0 and above. .IP \[bu] 2 -Run mergerfs as \f[C]root\f[] (with \f[B]allow_other\f[]) unless -you\[aq]re merging paths which are owned by the same user otherwise +Run mergerfs as \f[C]root\f[R] (with \f[B]allow_other\f[R]) unless +you\[cq]re merging paths which are owned by the same user otherwise strange permission issues may arise. .IP \[bu] 2 https://github.com/trapexit/backup\-and\-recovery\-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure. .IP \[bu] 2 -If you don\[aq]t see some directories and files you expect in a merged +If you don\[cq]t see some directories and files you expect in a merged point or policies seem to skip drives be sure the user has permission to all the underlying directories. -Use \f[C]mergerfs.fsck\f[] to audit the drive for out of sync +Use \f[C]mergerfs.fsck\f[R] to audit the drive for out of sync permissions. .IP \[bu] 2 -Do \f[B]not\f[] use \f[C]cache.files=off\f[] if you expect applications -(such as rtorrent) to use mmap (http://linux.die.net/man/2/mmap) files. +Do \f[B]not\f[R] use \f[C]cache.files=off\f[R] if you expect +applications (such as rtorrent) to use +mmap (http://linux.die.net/man/2/mmap) files. Shared mmap is not currently supported in FUSE w/ page caching disabled. -Enabling \f[C]dropcacheonclose\f[] is recommended when -\f[C]cache.files=partial|full|auto\-full\f[]. +Enabling \f[C]dropcacheonclose\f[R] is recommended when +\f[C]cache.files=partial|full|auto\-full\f[R]. .IP \[bu] 2 Kodi (http://kodi.tv), Plex (http://plex.tv), Subsonic (http://subsonic.org), etc. can use directory mtime (http://linux.die.net/man/2/stat) to more efficiently determine whether to scan for new content rather than simply performing a full scan. -If using the default \f[B]getattr\f[] policy of \f[B]ff\f[] it\[aq]s +If using the default \f[B]getattr\f[R] policy of \f[B]ff\f[R] it\[cq]s possible those programs will miss an update on account of it returning -the first directory found\[aq]s \f[B]stat\f[] info and its a later -directory on another mount which had the \f[B]mtime\f[] recently +the first directory found\[cq]s \f[B]stat\f[R] info and its a later +directory on another mount which had the \f[B]mtime\f[R] recently updated. -To fix this you will want to set \f[B]func.getattr=newest\f[]. -Remember though that this is just \f[B]stat\f[]. -If the file is later \f[B]open\f[]\[aq]ed or \f[B]unlink\f[]\[aq]ed and -the policy is different for those then a completely different file or -directory could be acted on. +To fix this you will want to set \f[B]func.getattr=newest\f[R]. +Remember though that this is just \f[B]stat\f[R]. +If the file is later \f[B]open\f[R]\[cq]ed or \f[B]unlink\f[R]\[cq]ed +and the policy is different for those then a completely different file +or directory could be acted on. .IP \[bu] 2 Some policies mixed with some functions may result in strange behaviors. -Not that some of these behaviors and race conditions couldn\[aq]t happen -outside \f[B]mergerfs\f[] but that they are far more likely to occur on +Not that some of these behaviors and race conditions couldn\[cq]t happen +outside \f[B]mergerfs\f[R] but that they are far more likely to occur on account of the attempt to merge together multiple sources of data which could be out of sync due to the different policies. .IP \[bu] 2 -For consistency its generally best to set \f[B]category\f[] wide -policies rather than individual \f[B]func\f[]\[aq]s. +For consistency its generally best to set \f[B]category\f[R] wide +policies rather than individual \f[B]func\f[R]\[cq]s. This will help limit the confusion of tools such as rsync (http://linux.die.net/man/1/rsync). However, the flexibility is there if needed. @@ -1954,9 +1986,9 @@ However, the flexibility is there if needed. .SS directory mtime is not being updated .PP -Remember that the default policy for \f[C]getattr\f[] is \f[C]ff\f[]. +Remember that the default policy for \f[C]getattr\f[R] is \f[C]ff\f[R]. The information for the first directory found will be returned. -If it wasn\[aq]t the directory which had been updated then it will +If it wasn\[cq]t the directory which had been updated then it will appear outdated. .PP The reason this is the default is because any other policy would be more @@ -1965,26 +1997,26 @@ To always return the directory with the most recent mtime or a faked value based on all found would require a scan of all drives. .PP If you always want the directory information from the one with the most -recent mtime then use the \f[C]newest\f[] policy for \f[C]getattr\f[]. -.SS \[aq]mv /mnt/pool/foo /mnt/disk1/foo\[aq] removes \[aq]foo\[aq] +recent mtime then use the \f[C]newest\f[R] policy for \f[C]getattr\f[R]. +.SS `mv /mnt/pool/foo /mnt/disk1/foo' removes `foo' .PP This is not a bug. .PP -Run in verbose mode to better understand what\[aq]s happening: +Run in verbose mode to better understand what\[cq]s happening: .IP .nf \f[C] -$\ mv\ \-v\ /mnt/pool/foo\ /mnt/disk1/foo -copied\ \[aq]/mnt/pool/foo\[aq]\ \->\ \[aq]/mnt/disk1/foo\[aq] -removed\ \[aq]/mnt/pool/foo\[aq] -$\ ls\ /mnt/pool/foo -ls:\ cannot\ access\ \[aq]/mnt/pool/foo\[aq]:\ No\ such\ file\ or\ directory -\f[] +$ mv \-v /mnt/pool/foo /mnt/disk1/foo +copied \[aq]/mnt/pool/foo\[aq] \-> \[aq]/mnt/disk1/foo\[aq] +removed \[aq]/mnt/pool/foo\[aq] +$ ls /mnt/pool/foo +ls: cannot access \[aq]/mnt/pool/foo\[aq]: No such file or directory +\f[R] .fi .PP -\f[C]mv\f[], when working across devices, is copying the source to +\f[C]mv\f[R], when working across devices, is copying the source to target and then removing the source. -Since the source \f[B]is\f[] the target in this case, depending on the +Since the source \f[B]is\f[R] the target in this case, depending on the unlink policy, it will remove the just copied file and other files across the branches. .PP @@ -1993,7 +2025,7 @@ mergerfs.dedup to clean up the old paths or manually remove them from the branches directly. .SS cached memory appears greater than it should be .PP -Use \f[C]cache.files=off\f[] and/or \f[C]dropcacheonclose=true\f[]. +Use \f[C]cache.files=off\f[R] and/or \f[C]dropcacheonclose=true\f[R]. See the section on page caching. .SS NFS clients returning ESTALE / Stale file handle .PP @@ -2009,113 +2041,114 @@ use_ino inodecalc=path\-hash .SS rtorrent fails with ENODEV (No such device) .PP -Be sure to set \f[C]cache.files=partial|full|auto\-full\f[] or turn off -\f[C]direct_io\f[]. +Be sure to set \f[C]cache.files=partial|full|auto\-full\f[R] or turn off +\f[C]direct_io\f[R]. rtorrent and some other applications use mmap (http://linux.die.net/man/2/mmap) to read and write to files and offer no fallback to traditional methods. -FUSE does not currently support mmap while using \f[C]direct_io\f[]. -There may be a performance penalty on writes with \f[C]direct_io\f[] off -as well as the problem of double caching but it\[aq]s the only way to -get such applications to work. +FUSE does not currently support mmap while using \f[C]direct_io\f[R]. +There may be a performance penalty on writes with \f[C]direct_io\f[R] +off as well as the problem of double caching but it\[cq]s the only way +to get such applications to work. If the performance loss is too high for other apps you can mount mergerfs twice. -Once with \f[C]direct_io\f[] enabled and one without it. -Be sure to set \f[C]dropcacheonclose=true\f[] if not using -\f[C]direct_io\f[]. -.SS Plex doesn\[aq]t work with mergerfs +Once with \f[C]direct_io\f[R] enabled and one without it. +Be sure to set \f[C]dropcacheonclose=true\f[R] if not using +\f[C]direct_io\f[R]. +.SS Plex doesn\[cq]t work with mergerfs .PP It does. -If you\[aq]re trying to put Plex\[aq]s config / metadata / database on -mergerfs you can\[aq]t set \f[C]cache.files=off\f[] because Plex is +If you\[cq]re trying to put Plex\[cq]s config / metadata / database on +mergerfs you can\[cq]t set \f[C]cache.files=off\f[R] because Plex is using sqlite3 with mmap enabled. -Shared mmap is not supported by Linux\[aq]s FUSE implementation when +Shared mmap is not supported by Linux\[cq]s FUSE implementation when page caching is disabled. To fix this place the data elsewhere (preferable) or enable -\f[C]cache.files\f[] (with \f[C]dropcacheonclose=true\f[]). +\f[C]cache.files\f[R] (with \f[C]dropcacheonclose=true\f[R]). Sqlite3 does not need mmap but the developer needs to fall back to standard IO if mmap fails. .PP -If the issue is that scanning doesn\[aq]t seem to pick up media then be -sure to set \f[C]func.getattr=newest\f[] though generally a full scan +If the issue is that scanning doesn\[cq]t seem to pick up media then be +sure to set \f[C]func.getattr=newest\f[R] though generally a full scan will pick up all media anyway. .SS When a program tries to move or rename a file it fails .PP -Please read the section above regarding rename & link (#rename--link). +Please read the section above regarding rename & link. .PP The problem is that many applications do not properly handle -\f[C]EXDEV\f[] errors which \f[C]rename\f[] and \f[C]link\f[] may return -even though they are perfectly valid situations which do not indicate -actual drive or OS errors. +\f[C]EXDEV\f[R] errors which \f[C]rename\f[R] and \f[C]link\f[R] may +return even though they are perfectly valid situations which do not +indicate actual drive or OS errors. The error will only be returned by mergerfs if using a path preserving policy as described in the policy section above. If you do not care about path preservation simply change the mergerfs policy to the non\-path preserving version. -For example: \f[C]\-o\ category.create=mfs\f[] +For example: \f[C]\-o category.create=mfs\f[R] .PP Ideally the offending software would be fixed and it is recommended that -if you run into this problem you contact the software\[aq]s author and -request proper handling of \f[C]EXDEV\f[] errors. +if you run into this problem you contact the software\[cq]s author and +request proper handling of \f[C]EXDEV\f[R] errors. .SS my 32bit software has problems .PP Some software have problems with 64bit inode values. The symptoms can include EOVERFLOW errors when trying to list files. -You can address this by setting \f[C]inodecalc\f[] to one of the 32bit +You can address this by setting \f[C]inodecalc\f[R] to one of the 32bit based algos as described in the relevant section. .SS Samba: Moving files / directories fails .PP Workaround: Copy the file/directory and then remove the original rather than move. .PP -This isn\[aq]t an issue with Samba but some SMB clients. +This isn\[cq]t an issue with Samba but some SMB clients. GVFS\-fuse v1.20.3 and prior (found in Ubuntu 14.04 among others) failed to handle certain error codes correctly. -Particularly \f[B]STATUS_NOT_SAME_DEVICE\f[] which comes from the -\f[B]EXDEV\f[] which is returned by \f[B]rename\f[] when the call is +Particularly \f[B]STATUS_NOT_SAME_DEVICE\f[R] which comes from the +\f[B]EXDEV\f[R] which is returned by \f[B]rename\f[R] when the call is crossing mount points. -When a program gets an \f[B]EXDEV\f[] it needs to explicitly take an +When a program gets an \f[B]EXDEV\f[R] it needs to explicitly take an alternate action to accomplish its goal. -In the case of \f[B]mv\f[] or similar it tries \f[B]rename\f[] and on -\f[B]EXDEV\f[] falls back to a manual copying of data between the two +In the case of \f[B]mv\f[R] or similar it tries \f[B]rename\f[R] and on +\f[B]EXDEV\f[R] falls back to a manual copying of data between the two locations and unlinking the source. -In these older versions of GVFS\-fuse if it received \f[B]EXDEV\f[] it -would translate that into \f[B]EIO\f[]. -This would cause \f[B]mv\f[] or most any application attempting to move +In these older versions of GVFS\-fuse if it received \f[B]EXDEV\f[R] it +would translate that into \f[B]EIO\f[R]. +This would cause \f[B]mv\f[R] or most any application attempting to move files around on that SMB share to fail with a IO error. .PP GVFS\-fuse v1.22.0 (https://bugzilla.gnome.org/show_bug.cgi?id=734568) and above fixed this issue but a large number of systems use the older release. On Ubuntu the version can be checked by issuing -\f[C]apt\-cache\ showpkg\ gvfs\-fuse\f[]. +\f[C]apt\-cache showpkg gvfs\-fuse\f[R]. Most distros released in 2015 seem to have the updated release and will work fine but older systems may not. Upgrading gvfs\-fuse or the distro in general will address the problem. .PP -In Apple\[aq]s MacOSX 10.9 they replaced Samba (client and server) with +In Apple\[cq]s MacOSX 10.9 they replaced Samba (client and server) with their own product. -It appears their new client does not handle \f[B]EXDEV\f[] either and +It appears their new client does not handle \f[B]EXDEV\f[R] either and responds similar to older release of gvfs on Linux. .SS Trashing files occasionally fails .PP This is the same issue as with Samba. -\f[C]rename\f[] returns \f[C]EXDEV\f[] (in our case that will really -only happen with path preserving policies like \f[C]epmfs\f[]) and the -software doesn\[aq]t handle the situation well. +\f[C]rename\f[R] returns \f[C]EXDEV\f[R] (in our case that will really +only happen with path preserving policies like \f[C]epmfs\f[R]) and the +software doesn\[cq]t handle the situation well. This is unfortunately a common failure of software which moves files around. -The standard indicates that an implementation \f[C]MAY\f[] choose to +The standard indicates that an implementation \f[C]MAY\f[R] choose to support non\-user home directory trashing of files (which is a -\f[C]MUST\f[]). -The implementation \f[C]MAY\f[] also support "top directory trashes" -which many probably do. -.PP -To create a \f[C]$topdir/.Trash\f[] directory as defined in the standard -use the mergerfs\-tools (https://github.com/trapexit/mergerfs-tools) -tool \f[C]mergerfs.mktrash\f[]. +\f[C]MUST\f[R]). +The implementation \f[C]MAY\f[R] also support \[lq]top directory +trashes\[rq] which many probably do. +.PP +To create a \f[C]$topdir/.Trash\f[R] directory as defined in the +standard use the +mergerfs\-tools (https://github.com/trapexit/mergerfs-tools) tool +\f[C]mergerfs.mktrash\f[R]. .SS tar: Directory renamed before its status could be extracted .PP -Make sure to use the \f[C]use_ino\f[] option. +Make sure to use the \f[C]use_ino\f[R] option. .SS Supplemental user groups .PP Due to the overhead of @@ -2127,14 +2160,14 @@ particular thread needs to change credentials and will keep that data for the lifetime of the thread. This means that if a user is added to a group it may not be picked up without the restart of mergerfs. -However, since the high level FUSE API\[aq]s (at least the standard -version) thread pool dynamically grows and shrinks it\[aq]s possible +However, since the high level FUSE API\[cq]s (at least the standard +version) thread pool dynamically grows and shrinks it\[cq]s possible that over time a thread will be killed and later a new thread with no cache will start and query the new data. .PP The gid cache uses fixed storage to simplify the design and be compatible with older systems which may not have C++11 compilers. -There is enough storage for 256 users\[aq] supplemental groups. +There is enough storage for 256 users\[cq] supplemental groups. Each user is allowed up to 32 supplemental groups. Linux >= 2.6.3 allows up to 65535 groups per user but most other *nixs allow far less. @@ -2143,18 +2176,18 @@ The system does handle overflow gracefully. If the user has more than 32 supplemental groups only the first 32 will be used. If more than 256 users are using the system when an uncached user is -found it will evict an existing user\[aq]s cache at random. -So long as there aren\[aq]t more than 256 active users this should be +found it will evict an existing user\[cq]s cache at random. +So long as there aren\[cq]t more than 256 active users this should be fine. If either value is too low for your needs you will have to modify -\f[C]gidcache.hpp\f[] to increase the values. +\f[C]gidcache.hpp\f[R] to increase the values. Note that doing so will increase the memory needed by each thread. .PP While not a bug some users have found when using containers that -supplemental groups defined inside the container don\[aq]t work properly +supplemental groups defined inside the container don\[cq]t work properly with regard to permissions. This is expected as mergerfs lives outside the container and therefore -is querying the host\[aq]s group database. +is querying the host\[cq]s group database. There might be a hack to work around this (make mergerfs read the /etc/group file in the container) but it is not yet implemented and would be limited to Linux and the /etc/group DB. @@ -2162,16 +2195,15 @@ Preferably users would mount in the host group file into the containers or use a standard shared user & groups technology like NIS or LDAP. .SS mergerfs or libfuse crashing .PP -First... -always upgrade to the latest version unless told otherwise. +First\&... always upgrade to the latest version unless told otherwise. .PP If using mergerfs below 2.22.0: .PP If suddenly the mergerfs mount point disappears and -\f[C]Transport\ endpoint\ is\ not\ connected\f[] is returned when -attempting to perform actions within the mount directory \f[B]and\f[] -the version of libfuse (use \f[C]mergerfs\ \-v\f[] to find the version) -is older than \f[C]2.9.4\f[] its likely due to a bug in libfuse. +\f[C]Transport endpoint is not connected\f[R] is returned when +attempting to perform actions within the mount directory \f[B]and\f[R] +the version of libfuse (use \f[C]mergerfs \-v\f[R] to find the version) +is older than \f[C]2.9.4\f[R] its likely due to a bug in libfuse. Affected versions of libfuse can be found in Debian Wheezy, Ubuntu Precise and others. .PP @@ -2188,10 +2220,10 @@ First upgrade if possible, check the known bugs section, and contact trapexit. .SS mergerfs appears to be crashing or exiting .PP -There seems to be an issue with Linux version \f[C]4.9.0\f[] and above +There seems to be an issue with Linux version \f[C]4.9.0\f[R] and above in which an invalid message appears to be transmitted to libfuse (used by mergerfs) causing it to exit. -No messages will be printed in any logs as it\[aq]s not a proper crash. +No messages will be printed in any logs as it\[cq]s not a proper crash. Debugging of the issue is still ongoing and can be followed via the fuse\-devel thread (https://sourceforge.net/p/fuse/mailman/message/35662577). @@ -2201,64 +2233,63 @@ Please update. This is only happened to mergerfs versions at or below v2.25.x and will not occur in more recent versions. .SH FAQ -.SS How well does mergerfs scale? Is it "production ready?" +.SS How well does mergerfs scale? Is it \[lq]production ready?\[rq] .PP Users have reported running mergerfs on everything from a Raspberry Pi to dual socket Xeon systems with >20 cores. -I\[aq]m aware of at least a few companies which use mergerfs in +I\[cq]m aware of at least a few companies which use mergerfs in production. Open Media Vault (https://www.openmediavault.org) includes mergerfs as its sole solution for pooling drives. The author of mergerfs had it running for over 300 days managing 16+ drives with reasonably heavy 24/7 read and write usage. -Stopping only after the machine\[aq]s power supply died. +Stopping only after the machine\[cq]s power supply died. .PP Most serious issues (crashes or data corruption) have been due to kernel bugs (https://github.com/trapexit/mergerfs/wiki/Kernel-Issues-&-Bugs). All of which are fixed in stable releases. -.SS Can mergerfs be used with drives which already have data / are in -use? +.SS Can mergerfs be used with drives which already have data / are in use? .PP Yes. -MergerFS is a proxy and does \f[B]NOT\f[] interfere with the normal form -or function of the drives / mounts / paths it manages. -.PP -MergerFS is \f[B]not\f[] a traditional filesystem. -MergerFS is \f[B]not\f[] RAID. -It does \f[B]not\f[] manipulate the data that passes through it. -It does \f[B]not\f[] shard data across drives. -It merely shards some \f[B]behavior\f[] and aggregates others. +MergerFS is a proxy and does \f[B]NOT\f[R] interfere with the normal +form or function of the drives / mounts / paths it manages. +.PP +MergerFS is \f[B]not\f[R] a traditional filesystem. +MergerFS is \f[B]not\f[R] RAID. +It does \f[B]not\f[R] manipulate the data that passes through it. +It does \f[B]not\f[R] shard data across drives. +It merely shards some \f[B]behavior\f[R] and aggregates others. .SS Can mergerfs be removed without affecting the data? .PP -See the previous question\[aq]s answer. +See the previous question\[cq]s answer. .SS What policies should I use? .PP -Unless you\[aq]re doing something more niche the average user is -probably best off using \f[C]mfs\f[] for \f[C]category.create\f[]. +Unless you\[cq]re doing something more niche the average user is +probably best off using \f[C]mfs\f[R] for \f[C]category.create\f[R]. It will spread files out across your branches based on available space. -Use \f[C]mspmfs\f[] if you want to try to colocate the data a bit more. -You may want to use \f[C]lus\f[] if you prefer a slightly different +Use \f[C]mspmfs\f[R] if you want to try to colocate the data a bit more. +You may want to use \f[C]lus\f[R] if you prefer a slightly different distribution of data if you have a mix of smaller and larger drives. -Generally though \f[C]mfs\f[], \f[C]lus\f[], or even \f[C]rand\f[] are -good for the general use case. +Generally though \f[C]mfs\f[R], \f[C]lus\f[R], or even \f[C]rand\f[R] +are good for the general use case. If you are starting with an imbalanced pool you can use the tool -\f[B]mergerfs.balance\f[] to redistribute files across the pool. +\f[B]mergerfs.balance\f[R] to redistribute files across the pool. .PP If you really wish to try to colocate files based on directory you can -set \f[C]func.create\f[] to \f[C]epmfs\f[] or similar and -\f[C]func.mkdir\f[] to \f[C]rand\f[] or \f[C]eprand\f[] depending on if -you just want to colocate generally or on specific branches. -Either way the \f[I]need\f[] to colocate is rare. +set \f[C]func.create\f[R] to \f[C]epmfs\f[R] or similar and +\f[C]func.mkdir\f[R] to \f[C]rand\f[R] or \f[C]eprand\f[R] depending on +if you just want to colocate generally or on specific branches. +Either way the \f[I]need\f[R] to colocate is rare. For instance: if you wish to remove the drive regularly and want the -data to predictably be on that drive or if you don\[aq]t use backup at -all and don\[aq]t wish to replace that data piecemeal. +data to predictably be on that drive or if you don\[cq]t use backup at +all and don\[cq]t wish to replace that data piecemeal. In which case using path preservation can help but will require some manual attention. Colocating after the fact can be accomplished using the -\f[B]mergerfs.consolidate\f[] tool. -If you don\[aq]t need strict colocation which the \f[C]ep\f[] policies -provide then you can use the \f[C]msp\f[] based policies which will walk -back the path till finding a branch that works. +\f[B]mergerfs.consolidate\f[R] tool. +If you don\[cq]t need strict colocation which the \f[C]ep\f[R] policies +provide then you can use the \f[C]msp\f[R] based policies which will +walk back the path till finding a branch that works. .PP Ultimately there is no correct answer. It is a preference or based on some particular need. @@ -2266,34 +2297,34 @@ mergerfs is very easy to test and experiment with. I suggest creating a test setup and experimenting to get a sense of what you want. .PP -The reason \f[C]mfs\f[] is not the default \f[C]category.create\f[] +The reason \f[C]mfs\f[R] is not the default \f[C]category.create\f[R] policy is historical. When/if a 3.X gets released it will be changed to minimize confusion people often have with path preserving policies. .SS What settings should I use? .PP Depends on what features you want. -Generally speaking there are no "wrong" settings. +Generally speaking there are no \[lq]wrong\[rq] settings. All settings are performance or feature related. The best bet is to read over the available options and choose what fits your situation. -If something isn\[aq]t clear from the documentation please reach out and +If something isn\[cq]t clear from the documentation please reach out and the documentation will be improved. .PP That said, for the average person, the following should be fine: .PP -\f[C]\-o\ use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs\f[] +\f[C]\-o use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs\f[R] .SS Why are all my files ending up on 1 drive?! .PP Did you start with empty drives? -Did you explicitly configure a \f[C]category.create\f[] policy? -Are you using an \f[C]existing\ path\f[] / \f[C]path\ preserving\f[] +Did you explicitly configure a \f[C]category.create\f[R] policy? +Are you using an \f[C]existing path\f[R] / \f[C]path preserving\f[R] policy? .PP -The default create policy is \f[C]epmfs\f[]. +The default create policy is \f[C]epmfs\f[R]. That is a path preserving algorithm. -With such a policy for \f[C]mkdir\f[] and \f[C]create\f[] with a set of -empty drives it will select only 1 drive when the first directory is +With such a policy for \f[C]mkdir\f[R] and \f[C]create\f[R] with a set +of empty drives it will select only 1 drive when the first directory is created. Anything, files or directories, created in that first directory will be placed on the same branch because it is preserving paths. @@ -2301,25 +2332,36 @@ placed on the same branch because it is preserving paths. This catches a lot of new users off guard but changing the default would break the setup for many existing users. If you do not care about path preservation and wish your files to be -spread across all your drives change to \f[C]mfs\f[] or similar policy +spread across all your drives change to \f[C]mfs\f[R] or similar policy as described above. -If you do want path preservation you\[aq]ll need to perform the manual +If you do want path preservation you\[cq]ll need to perform the manual act of creating paths on the drives you want the data to land on before transferring your data. -Setting \f[C]func.mkdir=epall\f[] can simplify managing path -preservation for \f[C]create\f[]. -Or use \f[C]func.mkdir=rand\f[] if you\[aq]re interested in just +Setting \f[C]func.mkdir=epall\f[R] can simplify managing path +preservation for \f[C]create\f[R]. +Or use \f[C]func.mkdir=rand\f[R] if you\[cq]re interested in just grouping together directory content by drive. -.SS Do hard links work? +.SS Do hardlinks work? .PP Yes. -You need to use \f[C]use_ino\f[] to support proper reporting of inodes +You need to use \f[C]use_ino\f[R] to support proper reporting of inodes but they work regardless. -See also the option \f[C]inodecalc\f[]. +See also the option \f[C]inodecalc\f[R]. .PP What mergerfs does not do is fake hard links across branches. -Read the section "rename & link" for how it works. -.SS Does mergerfs support CoW / copy\-on\-write? +Read the section \[lq]rename & link\[rq] for how it works. +.PP +Remember that hardlinks will NOT work across devices. +That includes between the original filesystem and a mergerfs pool, +between two separate pools of the same underlying filesystems, or bind +mounts of paths within the mergerfs pool. +The latter is common when using Docker or Podman. +Multiple volumes (bind mounts) to the same underlying filesystem are +considered different devices. +There is no way to link between them. +You should mount in the highest directory in the mergerfs pool that +includes all the paths you need if you want links to work. +.SS Does mergerfs support CoW / copy\-on\-write / writes to read\-only filesystems? .PP Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs or aufs sense. @@ -2329,9 +2371,13 @@ like hard link breaking (copy to temp file then rename over original) which can be useful when wanting to save space by hardlinking duplicate files but wish to treat each name as if it were a unique and separate file. -.SS Why can\[aq]t I see my files / directories? .PP -It\[aq]s almost always a permissions issue. +If you want to write to a read\-only filesystem you should look at +overlayfs. +You can always include the overlayfs mount into a mergerfs pool. +.SS Why can\[cq]t I see my files / directories? +.PP +It\[cq]s almost always a permissions issue. Unlike mhddfs and unionfs\-fuse, which runs as root and attempts to access content as such, mergerfs always changes its credentials to that of the caller. @@ -2345,12 +2391,12 @@ Whenever you run into a split permission issue (seeing some but not all files) try using mergerfs.fsck (https://github.com/trapexit/mergerfs-tools) tool to check for and fix the mismatch. -If you aren\[aq]t seeing anything at all be sure that the basic +If you aren\[cq]t seeing anything at all be sure that the basic permissions are correct. The user and group values are correct and that directories have their executable bit set. -A common mistake by users new to Linux is to \f[C]chmod\ \-R\ 644\f[] -when they should have \f[C]chmod\ \-R\ u=rwX,go=rX\f[]. +A common mistake by users new to Linux is to \f[C]chmod \-R 644\f[R] +when they should have \f[C]chmod \-R u=rwX,go=rX\f[R]. .PP If using a network filesystem such as NFS, SMB, CIFS (Samba) be sure to pay close attention to anything regarding permissioning and users. @@ -2358,28 +2404,27 @@ Root squashing and user translation for instance has bitten a few mergerfs users. Some of these also affect the use of mergerfs from container platforms such as Docker. -.SS Is my OS\[aq]s libfuse needed for mergerfs to work? -.PP -No. -Normally \f[C]mount.fuse\f[] is needed to get mergerfs (or any FUSE -filesystem to mount using the \f[C]mount\f[] command but in vendoring -the libfuse library the \f[C]mount.fuse\f[] app has been renamed to -\f[C]mount.mergerfs\f[] meaning the filesystem type in \f[C]fstab\f[] -can simply be \f[C]mergerfs\f[]. +.SS Is my OS\[cq]s libfuse needed for mergerfs to work? +.PP +No.\ Normally \f[C]mount.fuse\f[R] is needed to get mergerfs (or any +FUSE filesystem to mount using the \f[C]mount\f[R] command but in +vendoring the libfuse library the \f[C]mount.fuse\f[R] app has been +renamed to \f[C]mount.mergerfs\f[R] meaning the filesystem type in +\f[C]fstab\f[R] can simply be \f[C]mergerfs\f[R]. That said there should be no harm in having it installed and continuing -to using \f[C]fuse.mergerfs\f[] as the type in \f[C]/etc/fstab\f[]. +to using \f[C]fuse.mergerfs\f[R] as the type in \f[C]/etc/fstab\f[R]. .PP -If \f[C]mergerfs\f[] doesn\[aq]t work as a type it could be due to how -the \f[C]mount.mergerfs\f[] tool was installed. -Must be in \f[C]/sbin/\f[] with proper permissions. +If \f[C]mergerfs\f[R] doesn\[cq]t work as a type it could be due to how +the \f[C]mount.mergerfs\f[R] tool was installed. +Must be in \f[C]/sbin/\f[R] with proper permissions. .SS Why was libfuse embedded into mergerfs? .IP "1." 3 A significant number of users use mergerfs on distros with old versions of libfuse which have serious bugs. -Requiring updated versions of libfuse on those distros isn\[aq]t +Requiring updated versions of libfuse on those distros isn\[cq]t practical (no package offered, user inexperience, etc.). The only practical way to provide a stable runtime on those systems was -to "vendor" / embed the library into the project. +to \[lq]vendor\[rq] / embed the library into the project. .IP "2." 3 mergerfs was written to use the high level API. There are a number of limitations in the HLAPI that make certain @@ -2396,21 +2441,21 @@ Longer term the plan is to rewrite mergerfs to use the low level API. .PP See above first. .PP -If/when mergerfs is rewritten to use the low\-level API then it\[aq]ll +If/when mergerfs is rewritten to use the low\-level API then it\[cq]ll be plausible to support system libfuse but till then its simply too much work to manage the differences across the versions. .SS Why use mergerfs over mhddfs? .PP mhddfs is no longer maintained and has some known stability and security issues (see below). -MergerFS provides a superset of mhddfs\[aq] features and should offer +MergerFS provides a superset of mhddfs\[cq] features and should offer the same or maybe better performance. .PP Below is an example of mhddfs and mergerfs setup to work similarly. .PP -\f[C]mhddfs\ \-o\ mlimit=4G,allow_other\ /mnt/drive1,/mnt/drive2\ /mnt/pool\f[] +\f[C]mhddfs \-o mlimit=4G,allow_other /mnt/drive1,/mnt/drive2 /mnt/pool\f[R] .PP -\f[C]mergerfs\ \-o\ minfreespace=4G,allow_other,category.create=ff\ /mnt/drive1:/mnt/drive2\ /mnt/pool\f[] +\f[C]mergerfs \-o minfreespace=4G,allow_other,category.create=ff /mnt/drive1:/mnt/drive2 /mnt/pool\f[R] .SS Why use mergerfs over aufs? .PP aufs is mostly abandoned and no longer available in many distros. @@ -2423,14 +2468,13 @@ features which aufs and overlayfs have. .PP UnionFS is more like aufs than mergerfs in that it offers overlay / CoW features. -If you\[aq]re just looking to create a union of drives and want +If you\[cq]re just looking to create a union of drives and want flexibility in file/directory placement then mergerfs offers that whereas unionfs is more for overlaying RW filesystems over RO ones. .SS Why use mergerfs over overlayfs? .PP Same reasons as with unionfs. -.SS Why use mergerfs over LVM/ZFS/BTRFS/RAID0 drive concatenation / -striping? +.SS Why use mergerfs over LVM/ZFS/BTRFS/RAID0 drive concatenation / striping? .PP With simple JBOD / drive concatenation / stripping / RAID0 a single drive failure will result in full pool failure. @@ -2446,8 +2490,7 @@ without the single point of failure. MergerFS is not intended to be a replacement for ZFS. MergerFS is intended to provide flexible pooling of arbitrary drives (local or remote), of arbitrary sizes, and arbitrary filesystems. -For \f[C]write\ once,\ read\ many\f[] usecases such as bulk media -storage. +For \f[C]write once, read many\f[R] usecases such as bulk media storage. Where data integrity and backup is managed in other ways. In that situation ZFS can introduce a number of costs and limitations as described @@ -2463,14 +2506,14 @@ flexibility offered by mergerfs and for some the fact it is free and open source is important. .PP There are a number of UnRAID users who use mergerfs as well though -I\[aq]m not entirely familiar with the use case. +I\[cq]m not entirely familiar with the use case. .SS What should mergerfs NOT be used for? .IP \[bu] 2 databases: Even if the database stored data in separate files (mergerfs -wouldn\[aq]t offer much otherwise) the higher latency of the indirection +wouldn\[cq]t offer much otherwise) the higher latency of the indirection will kill performance. If it is a lightly used SQLITE database then it may be fine but -you\[aq]ll need to test. +you\[cq]ll need to test. .IP \[bu] 2 VM images: For the same reasons as databases. VM images are accessed very aggressively and mergerfs will introduce too @@ -2481,61 +2524,60 @@ If you need that kind of device performance aggregation or high availability you should stick with RAID. .SS Can drives be written to directly? Outside of mergerfs while pooled? .PP -Yes, however it\[aq]s not recommended to use the same file from within +Yes, however it\[cq]s not recommended to use the same file from within the pool and from without at the same time (particularly writing). Especially if using caching of any kind (cache.files, cache.entry, cache.attr, cache.negative_entry, cache.symlinks, cache.readdir, etc.) as there could be a conflict between cached data and not. -.SS Why do I get an "out of space" / "no space left on device" / ENOSPC -error even though there appears to be lots of space available? +.SS Why do I get an \[lq]out of space\[rq] / \[lq]no space left on device\[rq] / ENOSPC error even though there appears to be lots of space available? .PP -First make sure you\[aq]ve read the sections above about policies, path -preservation, branch filtering, and the options \f[B]minfreespace\f[], -\f[B]moveonenospc\f[], \f[B]statfs\f[], and \f[B]statfs_ignore\f[]. +First make sure you\[cq]ve read the sections above about policies, path +preservation, branch filtering, and the options \f[B]minfreespace\f[R], +\f[B]moveonenospc\f[R], \f[B]statfs\f[R], and \f[B]statfs_ignore\f[R]. .PP mergerfs is simply presenting a union of the content within multiple branches. The reported free space is an aggregate of space available within the -pool (behavior modified by \f[B]statfs\f[] and \f[B]statfs_ignore\f[]). +pool (behavior modified by \f[B]statfs\f[R] and +\f[B]statfs_ignore\f[R]). It does not represent a contiguous space. In the same way that read\-only filesystems, those with quotas, or reserved space report the full theoretical space available. .PP Due to path preservation, branch tagging, read\-only status, and -\f[B]minfreespace\f[] settings it is perfectly valid that -\f[C]ENOSPC\f[] / "out of space" / "no space left on device" be -returned. +\f[B]minfreespace\f[R] settings it is perfectly valid that +\f[C]ENOSPC\f[R] / \[lq]out of space\[rq] / \[lq]no space left on +device\[rq] be returned. It is doing what was asked of it: filtering possible branches due to those settings. Only one error can be returned and if one of the reasons for filtering a -branch was \f[B]minfreespace\f[] then it will be returned as such. -\f[B]moveonenospc\f[] is only relevant to writing a file which is too +branch was \f[B]minfreespace\f[R] then it will be returned as such. +\f[B]moveonenospc\f[R] is only relevant to writing a file which is too large for the drive its currently on. .PP It is also possible that the filesystem selected has run out of inodes. -Use \f[C]df\ \-i\f[] to list the total and available inodes per +Use \f[C]df \-i\f[R] to list the total and available inodes per filesystem. .PP -If you don\[aq]t care about path preservation then simply change the -\f[C]create\f[] policy to one which isn\[aq]t. -\f[C]mfs\f[] is probably what most are looking for. -The reason it\[aq]s not default is because it was originally set to -\f[C]epmfs\f[] and changing it now would change people\[aq]s setup. +If you don\[cq]t care about path preservation then simply change the +\f[C]create\f[R] policy to one which isn\[cq]t. +\f[C]mfs\f[R] is probably what most are looking for. +The reason it\[cq]s not default is because it was originally set to +\f[C]epmfs\f[R] and changing it now would change people\[cq]s setup. Such a setting change will likely occur in mergerfs 3. .SS Why does the total available space in mergerfs not equal outside? .PP Are you using ext2/3/4? With reserve for root? mergerfs uses available space for statfs calculations. -If you\[aq]ve reserved space for root then it won\[aq]t show up. +If you\[cq]ve reserved space for root then it won\[cq]t show up. .PP -You can remove the reserve by running: -\f[C]tune2fs\ \-m\ 0\ \f[] +You can remove the reserve by running: \f[C]tune2fs \-m 0 \f[R] .SS Can mergerfs mounts be exported over NFS? .PP Yes, however if you do anything which may changes files out of band -(including for example using the \f[C]newest\f[] policy) it will result -in "stale file handle" errors unless properly setup. +(including for example using the \f[C]newest\f[R] policy) it will result +in \[lq]stale file handle\[rq] errors unless properly setup. .PP Be sure to use the following options: .IP \[bu] 2 @@ -2554,52 +2596,51 @@ to how Samba is setup in relation to permissions. Yes. .SS I notice massive slowdowns of writes when enabling cache.files. .PP -When file caching is enabled in any form (\f[C]cache.files!=off\f[] or -\f[C]direct_io=false\f[]) it will issue \f[C]getxattr\f[] requests for -\f[C]security.capability\f[] prior to \f[I]every single write\f[]. +When file caching is enabled in any form (\f[C]cache.files!=off\f[R] or +\f[C]direct_io=false\f[R]) it will issue \f[C]getxattr\f[R] requests for +\f[C]security.capability\f[R] prior to \f[I]every single write\f[R]. This will usually result in a performance degradation, especially when using a network filesystem (such as NFS or CIFS/SMB/Samba.) Unfortunately at this moment the kernel is not caching the response. .PP To work around this situation mergerfs offers a few solutions. .IP "1." 3 -Set \f[C]security_capability=false\f[]. -It will short circuit any call and return \f[C]ENOATTR\f[]. +Set \f[C]security_capability=false\f[R]. +It will short circuit any call and return \f[C]ENOATTR\f[R]. This still means though that mergerfs will receive the request before -every write but at least it doesn\[aq]t get passed through to the +every write but at least it doesn\[cq]t get passed through to the underlying filesystem. .IP "2." 3 -Set \f[C]xattr=noattr\f[]. -Same as above but applies to \f[I]all\f[] calls to getxattr. -Not just \f[C]security.capability\f[]. -This will not be cached by the kernel either but mergerfs\[aq] runtime +Set \f[C]xattr=noattr\f[R]. +Same as above but applies to \f[I]all\f[R] calls to getxattr. +Not just \f[C]security.capability\f[R]. +This will not be cached by the kernel either but mergerfs\[cq] runtime config system will still function. .IP "3." 3 -Set \f[C]xattr=nosys\f[]. -Results in mergerfs returning \f[C]ENOSYS\f[] which \f[I]will\f[] be +Set \f[C]xattr=nosys\f[R]. +Results in mergerfs returning \f[C]ENOSYS\f[R] which \f[I]will\f[R] be cached by the kernel. No future xattr calls will be forwarded to mergerfs. The downside is that also means the xattr based config and query -functionality won\[aq]t work either. +functionality won\[cq]t work either. .IP "4." 3 Disable file caching. -If you aren\[aq]t using applications which use \f[C]mmap\f[] it\[aq]s +If you aren\[cq]t using applications which use \f[C]mmap\f[R] it\[cq]s probably simpler to just disable it all together. -The kernel won\[aq]t send the requests when caching is disabled. +The kernel won\[cq]t send the requests when caching is disabled. .SS What are these .fuse_hidden files? .PP Please upgrade. mergerfs >= 2.26.0 will not have these temporary files. -See the notes on \f[C]unlink\f[]. -.SS It\[aq]s mentioned that there are some security issues with mhddfs. -What are they? How does mergerfs address them? +See the notes on \f[C]unlink\f[R]. +.SS It\[cq]s mentioned that there are some security issues with mhddfs. What are they? How does mergerfs address them? .PP mhddfs (https://github.com/trapexit/mhddfs) manages running as -\f[B]root\f[] by calling +\f[B]root\f[R] by calling getuid() (https://github.com/trapexit/mhddfs/blob/cae96e6251dd91e2bdc24800b4a18a74044f6672/src/main.c#L319) -and if it returns \f[B]0\f[] then it will +and if it returns \f[B]0\f[R] then it will chown (http://linux.die.net/man/1/chown) the file. -Not only is that a race condition but it doesn\[aq]t handle other +Not only is that a race condition but it doesn\[cq]t handle other situations. Rather than attempting to simulate POSIX ACL behavior the proper way to manage this is to use seteuid (http://linux.die.net/man/2/seteuid) and @@ -2610,10 +2651,10 @@ This is what mergerfs does and why mergerfs should always run as root. In Linux setreuid syscalls apply only to the thread. GLIBC hides this away by using realtime signals to inform all threads to change credentials. -Taking after \f[B]Samba\f[], mergerfs uses -\f[B]syscall(SYS_setreuid,...)\f[] to set the callers credentials for +Taking after \f[B]Samba\f[R], mergerfs uses +\f[B]syscall(SYS_setreuid,\&...)\f[R] to set the callers credentials for that thread only. -Jumping back to \f[B]root\f[] as necessary should escalated privileges +Jumping back to \f[B]root\f[R] as necessary should escalated privileges be needed (for instance: to clone paths between drives). .PP For non\-Linux systems mergerfs uses a read\-write lock and changes @@ -2623,10 +2664,10 @@ to change the processes credentials. So long as the other threads need to be user X they will take a readlock allowing multiple threads to share the credentials. Once a request comes in to run as user Y that thread will attempt a -write lock and change to Y\[aq]s credentials when it can. +write lock and change to Y\[cq]s credentials when it can. If the ability to give writers priority is supported then that flag will -be used so threads trying to change credentials don\[aq]t starve. -This isn\[aq]t the best solution but should work reasonably well +be used so threads trying to change credentials don\[cq]t starve. +This isn\[cq]t the best solution but should work reasonably well assuming there are few users. .SH SUPPORT .PP @@ -2634,7 +2675,7 @@ Filesystems are complex and difficult to debug. mergerfs, while being just a proxy of sorts, is also very difficult to debug given the large number of possible settings it can have itself and the massive number of environments it can run in. -When reporting on a suspected issue \f[B]please, please\f[] include as +When reporting on a suspected issue \f[B]please, please\f[R] include as much of the below information as possible otherwise it will be difficult or impossible to diagnose. Also please make sure to read all of the above documentation as it @@ -2644,41 +2685,45 @@ includes nearly every known system or user issue previously encountered. release (https://github.com/trapexit/mergerfs/releases) or have tried it in comparison. Old versions, which are often included in distros like Debian and Ubuntu, are not ever going to be updated and your bug may -have been addressed already.\f[] +have been addressed already.\f[R] .SS Information to include in bug reports .IP \[bu] 2 -Version of mergerfs: \f[C]mergerfs\ \-V\f[] +Version of mergerfs: \f[C]mergerfs \-V\f[R] .IP \[bu] 2 -mergerfs settings: from \f[C]/etc/fstab\f[] or command line execution +mergerfs settings: from \f[C]/etc/fstab\f[R] or command line execution .IP \[bu] 2 -Version of Linux: \f[C]uname\ \-a\f[] +Version of Linux: \f[C]uname \-a\f[R] .IP \[bu] 2 Versions of any additional software being used .IP \[bu] 2 List of drives, their filesystems, and sizes (before and after issue): -\f[C]df\ \-h\f[] +\f[C]df \-h\f[R] .IP \[bu] 2 -\f[B]All\f[] information about the relevant branches and paths: +\f[B]All\f[R] information about the relevant branches and paths: permissions, etc. .IP \[bu] 2 -A \f[C]strace\f[] of the app having problems: +A \f[C]strace\f[R] of the app having problems: +.RS 2 .IP \[bu] 2 -\f[C]strace\ \-f\ \-o\ /tmp/app.strace.txt\ \f[] +\f[C]strace \-f \-o /tmp/app.strace.txt \f[R] +.RE .IP \[bu] 2 -A \f[C]strace\f[] of mergerfs while the program is trying to do whatever -it\[aq]s failing to do: +A \f[C]strace\f[R] of mergerfs while the program is trying to do +whatever it\[cq]s failing to do: +.RS 2 .IP \[bu] 2 -\f[C]strace\ \-f\ \-p\ \ \-o\ /tmp/mergerfs.strace.txt\f[] +\f[C]strace \-f \-p \-o /tmp/mergerfs.strace.txt\f[R] +.RE .IP \[bu] 2 -\f[B]Precise\f[] directions on replicating the issue. -Do not leave \f[B]anything\f[] out. +\f[B]Precise\f[R] directions on replicating the issue. +Do not leave \f[B]anything\f[R] out. .IP \[bu] 2 Try to recreate the problem in the simplest way using standard programs. .SS Contact / Issue submission .IP \[bu] 2 github.com: https://github.com/trapexit/mergerfs/issues .IP \[bu] 2 -email: trapexit\@spawn.link +email: trapexit\[at]spawn.link .IP \[bu] 2 twitter: https://twitter.com/_trapexit .IP \[bu] 2 From 3bc189cfc8a6cee8551accc1dfee97bd43950dfd Mon Sep 17 00:00:00 2001 From: Antonio SJ Musumeci Date: Sat, 23 Jan 2021 18:54:06 -0500 Subject: [PATCH 5/6] README: update support section --- README.md | 23 +- man/mergerfs.1 | 1629 ++++++++++++++++++++++++------------------------ 2 files changed, 826 insertions(+), 826 deletions(-) diff --git a/README.md b/README.md index da0ebada..5dd3aad3 100644 --- a/README.md +++ b/README.md @@ -1289,15 +1289,22 @@ At the moment my preference would be GitHub Sponsors only because I am part of t * SubscribeStar: https://www.subscribestar.com/trapexit * Ko-Fi: https://ko-fi.com/trapexit * Open Collective: https://opencollective.com/trapexit -* Bitcoin (BTC): 1DfoUd2m5WCxJAMvcFuvDpT4DR2gWX2PWb -* Bitcoin Cash (BCH): qrf257j0l09yxty4kur8dk2uma8p5vntdcpks72l8z -* Ethereum (ETH): 0xb486C0270fF75872Fc51d85879b9c15C380E66CA -* Litecoin (LTC): LW1rvHRPWtm2NUEMhJpP4DjHZY1FaJ1WYs -* Monero (XMR): 8AuU7PeK1fVhGP9yug8fdgKBssvUQoBVFKGhtT5DzWQt7fcTKC1SUx3Eb7xCAiVt3McWJp2Z9gX2wU7SPhh1GfWYBTCs6SS -* Basic Attention Token (BAT): 0xE651d4900B4C305284Da43E2e182e9abE149A87A + +* Bitcoin (BTC): bc1qjwlywkqxgrxql3m7a7fvcsf3z3t98jvtekqp2j +* Bitcoin Cash (BCH): qrvymmkvuk7703m7cx0pqxc3mz4mmsn6ngn9xw52kc +* Bitcoin SV (BSV): 1FkFuxRtt3f8LbkpeUKRZq7gKJFzGSGgZV +* Bitcoin Gold (BTG): Gfk8QbMJFgpMTcY7uB63axy6HU7uTPPWNj +* Basic Attention Token (BAT): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +* Chainlink (LINK): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +* Dash (DASH): Xu2U3Nd3G4hM5TRQUBcP4DHJFzXH93xB84 +* Dogecoin (DOGE): DGFBPsRBYL8wHbgnvKbYkVn5FvAe854p1c +* Ethereum (ETH): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +* Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy * LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r -* Zcash (ZEC): t1ZwTgmbQF23DJrzqbAmw8kXWvU2xUkkhTt -* Zcoin (XZC): a8L5Vz35KdCQe7Y7urK2pcCGau7JsqZ5Gw +* Litecoin (LTC): LfL7jLNYuVpy7v5TyRyc3yRZ2uhqc4UoR3 +* Monero (XMR): 45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f +* Tezos (XTZ): tz1ZxerkbbALsuU9XGV9K9fFpuLWnKAGfc1C +* Zcash (ZEC): t1bjbVBK7tx9EGBrnD2wDfjGV9yZrcyfMmr * Other crypto currencies: contact me for address diff --git a/man/mergerfs.1 b/man/mergerfs.1 index 5fa4e12a..12a3c2e8 100644 --- a/man/mergerfs.1 +++ b/man/mergerfs.1 @@ -1,5 +1,5 @@ .\"t -.\" Automatically generated by Pandoc 2.5 +.\" Automatically generated by Pandoc 1.19.2.4 .\" .TH "mergerfs" "1" "2020\-08\-30" "mergerfs user manual" "" .hy @@ -11,11 +11,10 @@ mergerfs \- a featureful union filesystem mergerfs \-o .SH DESCRIPTION .PP -\f[B]mergerfs\f[R] is a union filesystem geared towards simplifying +\f[B]mergerfs\f[] is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. -It is similar to \f[B]mhddfs\f[R], \f[B]unionfs\f[R], and -\f[B]aufs\f[R]. +It is similar to \f[B]mhddfs\f[], \f[B]unionfs\f[], and \f[B]aufs\f[]. .SH FEATURES .IP \[bu] 2 Runs in userspace (FUSE) @@ -54,33 +53,33 @@ Read more about policies below. .IP .nf \f[C] -A + B = C -/disk1 /disk2 /merged -| | | -+\-\- /dir1 +\-\- /dir1 +\-\- /dir1 -| | | | | | -| +\-\- file1 | +\-\- file2 | +\-\- file1 -| | +\-\- file3 | +\-\- file2 -+\-\- /dir2 | | +\-\- file3 -| | +\-\- /dir3 | -| +\-\- file4 | +\-\- /dir2 -| +\-\- file5 | | -+\-\- file6 | +\-\- file4 - | - +\-\- /dir3 - | | - | +\-\- file5 - | - +\-\- file6 -\f[R] +A\ \ \ \ \ \ \ \ \ +\ \ \ \ \ \ B\ \ \ \ \ \ \ \ =\ \ \ \ \ \ \ C +/disk1\ \ \ \ \ \ \ \ \ \ \ /disk2\ \ \ \ \ \ \ \ \ \ \ /merged +|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | ++\-\-\ /dir1\ \ \ \ \ \ \ \ +\-\-\ /dir1\ \ \ \ \ \ \ \ +\-\-\ /dir1 +|\ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ | +|\ \ \ +\-\-\ file1\ \ \ \ |\ \ \ +\-\-\ file2\ \ \ \ |\ \ \ +\-\-\ file1 +|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file3\ \ \ \ |\ \ \ +\-\-\ file2 ++\-\-\ /dir2\ \ \ \ \ \ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file3 +|\ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ /dir3\ \ \ \ \ \ \ \ | +|\ \ \ +\-\-\ file4\ \ \ \ \ \ \ \ |\ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ /dir2 +|\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ file5\ \ \ |\ \ \ | ++\-\-\ file6\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file4 +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ /dir3 +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ | +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ |\ \ \ +\-\-\ file5 +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ | +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +\-\-\ file6 +\f[] .fi .PP -mergerfs does \f[B]NOT\f[R] support the copy\-on\-write (CoW) or -whiteout behaviors found in \f[B]aufs\f[R] and \f[B]overlayfs\f[R]. -You can \f[B]not\f[R] mount a read\-only filesystem and write to it. +mergerfs does \f[B]NOT\f[] support the copy\-on\-write (CoW) or whiteout +behaviors found in \f[B]aufs\f[] and \f[B]overlayfs\f[]. +You can \f[B]not\f[] mount a read\-only filesystem and write to it. However, mergerfs will ignore read\-only drives when creating new files so you can mix read\-write and read\-only drives. -It also does \f[B]NOT\f[R] split data across drives. +It also does \f[B]NOT\f[] split data across drives. It is not RAID0 / striping. It is simply a union of other filesystems. .SH TERMINOLOGY @@ -103,14 +102,15 @@ path preservation: Aspect of some policies which includes checking the path for which a file would be created. .SH BASIC SETUP .PP -If you don\[cq]t already know that you have a special use case then just +If you don\[aq]t already know that you have a special use case then just start with one of the following option sets. -.SS You need \f[C]mmap\f[R] (used by rtorrent and many sqlite3 base software) +.SS You need \f[C]mmap\f[] (used by rtorrent and many sqlite3 base +software) .PP -\f[C]allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs\f[R] -.SS You don\[cq]t need \f[C]mmap\f[R] +\f[C]allow_other,use_ino,cache.files=partial,dropcacheonclose=true,category.create=mfs\f[] +.SS You don\[aq]t need \f[C]mmap\f[] .PP -\f[C]allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs\f[R] +\f[C]allow_other,use_ino,cache.files=off,dropcacheonclose=true,category.create=mfs\f[] .PP See the mergerfs wiki for real world deployments (https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments) @@ -118,125 +118,127 @@ for comparisons / ideas. .SH OPTIONS .PP These options are the same regardless you use them with the -\f[C]mergerfs\f[R] commandline program, used in fstab, or in a config +\f[C]mergerfs\f[] commandline program, used in fstab, or in a config file. .SS mount options .IP \[bu] 2 -\f[B]config\f[R]: Path to a config file. +\f[B]config\f[]: Path to a config file. Same arguments as below in key=val / ini style format. .IP \[bu] 2 -\f[B]branches\f[R]: Colon delimited list of branches. +\f[B]branches\f[]: Colon delimited list of branches. .IP \[bu] 2 -\f[B]allow_other\f[R]: A libfuse option which allows users besides the +\f[B]allow_other\f[]: A libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use\-cases. .IP \[bu] 2 -\f[B]minfreespace=SIZE\f[R]: The minimum space value used for creation +\f[B]minfreespace=SIZE\f[]: The minimum space value used for creation policies. Can be overridden by branch specific option. -Understands `K', `M', and `G' to represent kilobyte, megabyte, and -gigabyte respectively. +Understands \[aq]K\[aq], \[aq]M\[aq], and \[aq]G\[aq] to represent +kilobyte, megabyte, and gigabyte respectively. (default: 4G) .IP \[bu] 2 -\f[B]moveonenospc=BOOL|POLICY\f[R]: When enabled if a \f[B]write\f[R] -fails with \f[B]ENOSPC\f[R] (no space left on device) or -\f[B]EDQUOT\f[R] (disk quota exceeded) the policy selected will run to -find a new location for the file. +\f[B]moveonenospc=BOOL|POLICY\f[]: When enabled if a \f[B]write\f[] +fails with \f[B]ENOSPC\f[] (no space left on device) or \f[B]EDQUOT\f[] +(disk quota exceeded) the policy selected will run to find a new +location for the file. An attempt to move the file to that branch will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false, true = mfs) .IP \[bu] 2 -\f[B]use_ino\f[R]: Causes mergerfs to supply file/directory inodes -rather than libfuse. +\f[B]use_ino\f[]: Causes mergerfs to supply file/directory inodes rather +than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value. .IP \[bu] 2 -\f[B]inodecalc=passthrough|path\-hash|devino\-hash|hybrid\-hash\f[R]: +\f[B]inodecalc=passthrough|path\-hash|devino\-hash|hybrid\-hash\f[]: Selects the inode calculation algorithm. (default: hybrid\-hash) .IP \[bu] 2 -\f[B]dropcacheonclose=BOOL\f[R]: When a file is requested to be closed -call \f[C]posix_fadvise\f[R] on it first to instruct the kernel that we +\f[B]dropcacheonclose=BOOL\f[]: When a file is requested to be closed +call \f[C]posix_fadvise\f[] on it first to instruct the kernel that we no longer need the data and it can drop its cache. -Recommended when \f[B]cache.files=partial|full|auto\-full\f[R] to limit +Recommended when \f[B]cache.files=partial|full|auto\-full\f[] to limit double caching. (default: false) .IP \[bu] 2 -\f[B]symlinkify=BOOL\f[R]: When enabled and a file is not writable and -its mtime or ctime is older than \f[B]symlinkify_timeout\f[R] files will +\f[B]symlinkify=BOOL\f[]: When enabled and a file is not writable and +its mtime or ctime is older than \f[B]symlinkify_timeout\f[] files will be reported as symlinks to the original files. Please read more below before using. (default: false) .IP \[bu] 2 -\f[B]symlinkify_timeout=UINT\f[R]: Time to wait, in seconds, to activate -the \f[B]symlinkify\f[R] behavior. +\f[B]symlinkify_timeout=UINT\f[]: Time to wait, in seconds, to activate +the \f[B]symlinkify\f[] behavior. (default: 3600) .IP \[bu] 2 -\f[B]nullrw=BOOL\f[R]: Turns reads and writes into no\-ops. +\f[B]nullrw=BOOL\f[]: Turns reads and writes into no\-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false) .IP \[bu] 2 -\f[B]ignorepponrename=BOOL\f[R]: Ignore path preserving on rename. +\f[B]ignorepponrename=BOOL\f[]: Ignore path preserving on rename. Typically rename and link act differently depending on the policy of -\f[C]create\f[R] (read below). +\f[C]create\f[] (read below). Enabling this will cause rename and link to always use the non\-path preserving behavior. This means files, when renamed or linked, will stay on the same drive. (default: false) .IP \[bu] 2 -\f[B]security_capability=BOOL\f[R]: If false return ENOATTR when xattr +\f[B]security_capability=BOOL\f[]: If false return ENOATTR when xattr security.capability is queried. (default: true) .IP \[bu] 2 -\f[B]xattr=passthrough|noattr|nosys\f[R]: Runtime control of xattrs. +\f[B]xattr=passthrough|noattr|nosys\f[]: Runtime control of xattrs. Default is to passthrough xattr requests. -`noattr' will short circuit as if nothing exists. -`nosys' will respond with ENOSYS as if xattrs are not supported or -disabled. +\[aq]noattr\[aq] will short circuit as if nothing exists. +\[aq]nosys\[aq] will respond with ENOSYS as if xattrs are not supported +or disabled. (default: passthrough) .IP \[bu] 2 -\f[B]link_cow=BOOL\f[R]: When enabled if a regular file is opened which +\f[B]link_cow=BOOL\f[]: When enabled if a regular file is opened which has a link count > 1 it will copy the file to a temporary file and rename over the original. Breaking the link and providing a basic copy\-on\-write function similar to cow\-shell. (default: false) .IP \[bu] 2 -\f[B]statfs=base|full\f[R]: Controls how statfs works. -`base' means it will always use all branches in statfs calculations. -`full' is in effect path preserving and only includes drives where the -path exists. +\f[B]statfs=base|full\f[]: Controls how statfs works. +\[aq]base\[aq] means it will always use all branches in statfs +calculations. +\[aq]full\[aq] is in effect path preserving and only includes drives +where the path exists. (default: base) .IP \[bu] 2 -\f[B]statfs_ignore=none|ro|nc\f[R]: `ro' will cause statfs calculations -to ignore available space for branches mounted or tagged as `read\-only' -or `no create'. -`nc' will ignore available space for branches tagged as `no create'. +\f[B]statfs_ignore=none|ro|nc\f[]: \[aq]ro\[aq] will cause statfs +calculations to ignore available space for branches mounted or tagged as +\[aq]read\-only\[aq] or \[aq]no create\[aq]. +\[aq]nc\[aq] will ignore available space for branches tagged as \[aq]no +create\[aq]. (default: none) .IP \[bu] 2 -\f[B]nfsopenhack=off|git|all\f[R]: A workaround for exporting mergerfs +\f[B]nfsopenhack=off|git|all\f[]: A workaround for exporting mergerfs over NFS where there are issues with creating files for write while setting the mode to read\-only. (default: off) .IP \[bu] 2 -\f[B]posix_acl=BOOL\f[R]: Enable POSIX ACL support (if supported by +\f[B]posix_acl=BOOL\f[]: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false) .IP \[bu] 2 -\f[B]async_read=BOOL\f[R]: Perform reads asynchronously. +\f[B]async_read=BOOL\f[]: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true) .IP \[bu] 2 -\f[B]fuse_msg_size=UINT\f[R]: Set the max number of pages per FUSE +\f[B]fuse_msg_size=UINT\f[]: Set the max number of pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256) .IP \[bu] 2 -\f[B]threads=INT\f[R]: Number of threads to use in multithreaded mode. +\f[B]threads=INT\f[]: Number of threads to use in multithreaded mode. When set to zero it will attempt to discover and use the number of logical cores. If the lookup fails it will fall back to using 4. @@ -249,89 +251,90 @@ NOTE: higher number of threads increases parallelism but usually decreases throughput. (default: 0) .IP \[bu] 2 -\f[B]fsname=STR\f[R]: Sets the name of the filesystem as seen in -\f[B]mount\f[R], \f[B]df\f[R], etc. +\f[B]fsname=STR\f[]: Sets the name of the filesystem as seen in +\f[B]mount\f[], \f[B]df\f[], etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed. .IP \[bu] 2 -\f[B]func.FUNC=POLICY\f[R]: Sets the specific FUSE function\[cq]s -policy. +\f[B]func.FUNC=POLICY\f[]: Sets the specific FUSE function\[aq]s policy. See below for the list of value types. -Example: \f[B]func.getattr=newest\f[R] +Example: \f[B]func.getattr=newest\f[] .IP \[bu] 2 -\f[B]category.action=POLICY\f[R]: Sets policy of all FUSE functions in +\f[B]category.action=POLICY\f[]: Sets policy of all FUSE functions in the action category. (default: epall) .IP \[bu] 2 -\f[B]category.create=POLICY\f[R]: Sets policy of all FUSE functions in +\f[B]category.create=POLICY\f[]: Sets policy of all FUSE functions in the create category. (default: epmfs) .IP \[bu] 2 -\f[B]category.search=POLICY\f[R]: Sets policy of all FUSE functions in +\f[B]category.search=POLICY\f[]: Sets policy of all FUSE functions in the search category. (default: ff) .IP \[bu] 2 -\f[B]cache.open=UINT\f[R]: `open' policy cache timeout in seconds. +\f[B]cache.open=UINT\f[]: \[aq]open\[aq] policy cache timeout in +seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.statfs=UINT\f[R]: `statfs' cache timeout in seconds. +\f[B]cache.statfs=UINT\f[]: \[aq]statfs\[aq] cache timeout in seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.attr=UINT\f[R]: File attribute cache timeout in seconds. +\f[B]cache.attr=UINT\f[]: File attribute cache timeout in seconds. (default: 1) .IP \[bu] 2 -\f[B]cache.entry=UINT\f[R]: File name lookup cache timeout in seconds. +\f[B]cache.entry=UINT\f[]: File name lookup cache timeout in seconds. (default: 1) .IP \[bu] 2 -\f[B]cache.negative_entry=UINT\f[R]: Negative file name lookup cache +\f[B]cache.negative_entry=UINT\f[]: Negative file name lookup cache timeout in seconds. (default: 0) .IP \[bu] 2 -\f[B]cache.files=libfuse|off|partial|full|auto\-full\f[R]: File page +\f[B]cache.files=libfuse|off|partial|full|auto\-full\f[]: File page caching mode (default: libfuse) .IP \[bu] 2 -\f[B]cache.writeback=BOOL\f[R]: Enable kernel writeback caching -(default: false) +\f[B]cache.writeback=BOOL\f[]: Enable kernel writeback caching (default: +false) .IP \[bu] 2 -\f[B]cache.symlinks=BOOL\f[R]: Cache symlinks (if supported by kernel) +\f[B]cache.symlinks=BOOL\f[]: Cache symlinks (if supported by kernel) (default: false) .IP \[bu] 2 -\f[B]cache.readdir=BOOL\f[R]: Cache readdir (if supported by kernel) +\f[B]cache.readdir=BOOL\f[]: Cache readdir (if supported by kernel) (default: false) .IP \[bu] 2 -\f[B]direct_io\f[R]: deprecated \- Bypass page cache. -Use \f[C]cache.files=off\f[R] instead. +\f[B]direct_io\f[]: deprecated \- Bypass page cache. +Use \f[C]cache.files=off\f[] instead. (default: false) .IP \[bu] 2 -\f[B]kernel_cache\f[R]: deprecated \- Do not invalidate data cache on +\f[B]kernel_cache\f[]: deprecated \- Do not invalidate data cache on file open. -Use \f[C]cache.files=full\f[R] instead. +Use \f[C]cache.files=full\f[] instead. (default: false) .IP \[bu] 2 -\f[B]auto_cache\f[R]: deprecated \- Invalidate data cache if file mtime +\f[B]auto_cache\f[]: deprecated \- Invalidate data cache if file mtime or size change. -Use \f[C]cache.files=auto\-full\f[R] instead. +Use \f[C]cache.files=auto\-full\f[] instead. (default: false) .IP \[bu] 2 -\f[B]async_read\f[R]: deprecated \- Perform reads asynchronously. -Use \f[C]async_read=true\f[R] instead. +\f[B]async_read\f[]: deprecated \- Perform reads asynchronously. +Use \f[C]async_read=true\f[] instead. .IP \[bu] 2 -\f[B]sync_read\f[R]: deprecated \- Perform reads synchronously. -Use \f[C]async_read=false\f[R] instead. +\f[B]sync_read\f[]: deprecated \- Perform reads synchronously. +Use \f[C]async_read=false\f[] instead. .PP -\f[B]NOTE:\f[R] Options are evaluated in the order listed so if the -options are \f[B]func.rmdir=rand,category.action=ff\f[R] the -\f[B]action\f[R] category setting will override the \f[B]rmdir\f[R] +\f[B]NOTE:\f[] Options are evaluated in the order listed so if the +options are \f[B]func.rmdir=rand,category.action=ff\f[] the +\f[B]action\f[] category setting will override the \f[B]rmdir\f[] setting. .SS Value Types .IP \[bu] 2 -BOOL = `true' | `false' +BOOL = \[aq]true\[aq] | \[aq]false\[aq] .IP \[bu] 2 INT = [MIN_INT,MAX_INT] .IP \[bu] 2 UINT = [0,MAX_INT] .IP \[bu] 2 -SIZE = `NNM'; NN = INT, M = `K' | `M' | `G' | `T' +SIZE = \[aq]NNM\[aq]; NN = INT, M = \[aq]K\[aq] | \[aq]M\[aq] | +\[aq]G\[aq] | \[aq]T\[aq] .IP \[bu] 2 STR = string .IP \[bu] 2 @@ -342,19 +345,19 @@ CATEGORY = function category POLICY = mergerfs function policy .SS branches .PP -The `branches' argument is a colon (`:') delimited list of paths to be -pooled together. +The \[aq]branches\[aq] argument is a colon (\[aq]:\[aq]) delimited list +of paths to be pooled together. It does not matter if the paths are on the same or different drives nor does it matter the filesystem (within reason). Used and available space will not be duplicated for paths on the same -device and any features which aren\[cq]t supported by the underlying +device and any features which aren\[aq]t supported by the underlying filesystem (such as file attributes or extended attributes) will return the appropriate errors. .PP Branches currently have two options which can be set. A type which impacts whether or not the branch is included in a policy calculation and a individual minfreespace value. -The values are set by prepending an \f[C]=\f[R] at the end of a branch +The values are set by prepending an \f[C]=\f[] at the end of a branch designation and using commas as delimiters. Example: /mnt/drive=RW,1234 .SS branch type @@ -362,13 +365,13 @@ Example: /mnt/drive=RW,1234 RW: (read/write) \- Default behavior. Will be eligible in all policy categories. .IP \[bu] 2 -RO: (read\-only) \- Will be excluded from \f[C]create\f[R] and -\f[C]action\f[R] policies. +RO: (read\-only) \- Will be excluded from \f[C]create\f[] and +\f[C]action\f[] policies. Same as a read\-only mounted filesystem would be (though faster to process). .IP \[bu] 2 -NC: (no\-create) \- Will be excluded from \f[C]create\f[R] policies. -You can\[cq]t create on that branch but you can change or delete. +NC: (no\-create) \- Will be excluded from \f[C]create\f[] policies. +You can\[aq]t create on that branch but you can change or delete. .SS minfreespace .PP Same purpose as the global option but specific to the branch. @@ -378,35 +381,35 @@ If not set the global value is used. To make it easier to include multiple branches mergerfs supports globbing (http://linux.die.net/man/7/glob). \f[B]The globbing tokens MUST be escaped when using via the shell else -the shell itself will apply the glob itself.\f[R] +the shell itself will apply the glob itself.\f[] .IP .nf \f[C] -# mergerfs \-o allow_other,use_ino /mnt/disk\[rs]*:/mnt/cdrom /media/drives -\f[R] +#\ mergerfs\ \-o\ allow_other,use_ino\ /mnt/disk\\*:/mnt/cdrom\ /media/drives +\f[] .fi .PP The above line will use all mount points in /mnt prefixed with -\f[B]disk\f[R] and the \f[B]cdrom\f[R]. +\f[B]disk\f[] and the \f[B]cdrom\f[]. .PP To have the pool mounted at boot or otherwise accessible from related -tools use \f[B]/etc/fstab\f[R]. +tools use \f[B]/etc/fstab\f[]. .IP .nf \f[C] -# -/mnt/disk*:/mnt/cdrom /mnt/pool fuse.mergerfs allow_other,use_ino 0 0 -\f[R] +#\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ +/mnt/disk*:/mnt/cdrom\ \ /mnt/pool\ \ \ \ \ \ fuse.mergerfs\ \ allow_other,use_ino\ \ \ 0\ \ \ \ \ \ \ 0 +\f[] .fi .PP -\f[B]NOTE:\f[R] the globbing is done at mount or when updated using the +\f[B]NOTE:\f[] the globbing is done at mount or when updated using the runtime API. If a new directory is added matching the glob after the fact it will not be automatically included. .PP -\f[B]NOTE:\f[R] for mounting via \f[B]fstab\f[R] to work you must have -\f[B]mount.fuse\f[R] installed. -For Ubuntu/Debian it is included in the \f[B]fuse\f[R] package. +\f[B]NOTE:\f[] for mounting via \f[B]fstab\f[] to work you must have +\f[B]mount.fuse\f[] installed. +For Ubuntu/Debian it is included in the \f[B]fuse\f[] package. .SS inodecalc .PP Inodes (st_ino) are unique identifiers within a filesystem. @@ -420,14 +423,14 @@ the complexity they add. .PP FUSE allows the server (mergerfs) to set inode values but not device IDs. -Creating an inode value is somewhat complex in mergerfs\[cq] case as -files aren\[cq]t really in its control. +Creating an inode value is somewhat complex in mergerfs\[aq] case as +files aren\[aq]t really in its control. If a policy changes what directory or file is to be selected or something changes out of band it becomes unclear what value should be used. Most software does not to care what the values are but those that do often break if a value changes unexpectedly. -The tool \f[C]find\f[R] will abort a directory walk if it sees a +The tool \f[C]find\f[] will abort a directory walk if it sees a directory inode change. NFS will return stale handle errors if the inode changes out of band. File dedup tools will usually leverage device ids and inodes as a @@ -443,27 +446,27 @@ problems mentioned above and could confuse file deduplication software as inodes from different filesystems can be the same. .IP \[bu] 2 path\-hash: Hashes the relative path of the entry in question. -The underlying file\[cq]s values are completely ignored. +The underlying file\[aq]s values are completely ignored. This means the inode value will always be the same for that file path. This is useful when using NFS and you make changes out of band such as copy data between branches. This also means that entries that do point to the same file will not be recognizable via inodes. -That \f[B]does not\f[R] mean hard links don\[cq]t work. +That \f[B]does not\f[] mean hard links don\[aq]t work. They will. .IP \[bu] 2 path\-hash32: 32bit version of path\-hash. .IP \[bu] 2 devino\-hash: Hashes the device id and inode of the underlying entry. -This won\[cq]t prevent issues with NFS should the policy pick a +This won\[aq]t prevent issues with NFS should the policy pick a different file or files move out of band but will present the same inode for underlying files that do too. .IP \[bu] 2 devino\-hash32: 32bit version of devino\-hash. .IP \[bu] 2 -hybrid\-hash: Performs \f[C]path\-hash\f[R] on directories and -\f[C]devino\-hash\f[R] on other file types. -Since directories can\[cq]t have hard links the static value won\[cq]t +hybrid\-hash: Performs \f[C]path\-hash\f[] on directories and +\f[C]devino\-hash\f[] on other file types. +Since directories can\[aq]t have hard links the static value won\[aq]t make a difference and the files will get values useful for finding duplicates. Probably the best to use if not using NFS. @@ -480,36 +483,36 @@ Unlike a typical filesystem FUSE filesystems can reuse inodes and not refer to the same entry. The internal identifier used to reference a file in FUSE is different from the inode value presented. -The former is the \f[C]nodeid\f[R] and is actually a tuple of 2 64bit -values: \f[C]nodeid\f[R] and \f[C]generation\f[R]. +The former is the \f[C]nodeid\f[] and is actually a tuple of 2 64bit +values: \f[C]nodeid\f[] and \f[C]generation\f[]. This tuple is not client facing. The inode that is presented to the client is passed through the kernel uninterpreted. .PP -From FUSE docs regarding \f[C]use_ino\f[R]: +From FUSE docs regarding \f[C]use_ino\f[]: .IP .nf \f[C] -Honor the st_ino field in the functions getattr() and -fill_dir(). This value is used to fill in the st_ino field -in the stat(2), lstat(2), fstat(2) functions and the d_ino -field in the readdir(2) function. The filesystem does not -have to guarantee uniqueness, however some applications -rely on this value being unique for the whole filesystem. -Note that this does *not* affect the inode that libfuse -and the kernel use internally (also called the \[dq]nodeid\[dq]). -\f[R] +Honor\ the\ st_ino\ field\ in\ the\ functions\ getattr()\ and +fill_dir().\ This\ value\ is\ used\ to\ fill\ in\ the\ st_ino\ field +in\ the\ stat(2),\ lstat(2),\ fstat(2)\ functions\ and\ the\ d_ino +field\ in\ the\ readdir(2)\ function.\ The\ filesystem\ does\ not +have\ to\ guarantee\ uniqueness,\ however\ some\ applications +rely\ on\ this\ value\ being\ unique\ for\ the\ whole\ filesystem. +Note\ that\ this\ does\ *not*\ affect\ the\ inode\ that\ libfuse +and\ the\ kernel\ use\ internally\ (also\ called\ the\ "nodeid"). +\f[] .fi .PP -In the future the \f[C]use_ino\f[R] option will probably be removed as +In the future the \f[C]use_ino\f[] option will probably be removed as this feature should replace the original libfuse inode calculation strategy. -Currently you still need to use \f[C]use_ino\f[R] in order to enable -\f[C]inodecalc\f[R]. +Currently you still need to use \f[C]use_ino\f[] in order to enable +\f[C]inodecalc\f[]. .SS fuse_msg_size .PP FUSE applications communicate with the kernel over a special character -device: \f[C]/dev/fuse\f[R]. +device: \f[C]/dev/fuse\f[]. A large portion of the overhead associated with FUSE is the cost of going back and forth from user space and kernel space over that device. Generally speaking the fewer trips needed the better the performance @@ -524,16 +527,16 @@ In Linux 4.20 a new feature was added allowing the negotiation of the max message size. Since the size is in multiples of pages (https://en.wikipedia.org/wiki/Page_(computer_memory)) the feature -is called \f[C]max_pages\f[R]. -There is a maximum \f[C]max_pages\f[R] value of 256 (1MiB) and minimum -of 1 (4KiB). +is called \f[C]max_pages\f[]. +There is a maximum \f[C]max_pages\f[] value of 256 (1MiB) and minimum of +1 (4KiB). The default used by Linux >=4.20, and hardcoded value used before 4.20, is 32 (128KiB). -In mergerfs its referred to as \f[C]fuse_msg_size\f[R] to make it clear +In mergerfs its referred to as \f[C]fuse_msg_size\f[] to make it clear what it impacts and provide some abstraction. .PP -Since there should be no downsides to increasing \f[C]fuse_msg_size\f[R] -/ \f[C]max_pages\f[R], outside a minor bump in RAM usage due to larger +Since there should be no downsides to increasing \f[C]fuse_msg_size\f[] +/ \f[C]max_pages\f[], outside a minor bump in RAM usage due to larger message buffers, mergerfs defaults the value to 256. On kernels before 4.20 the value has no effect. The reason the value is configurable is to enable experimentation and @@ -545,16 +548,16 @@ Due to the levels of indirection introduced by mergerfs and the underlying technology FUSE there can be varying levels of performance degradation. This feature will turn non\-directories which are not writable into -symlinks to the original file found by the \f[C]readlink\f[R] policy +symlinks to the original file found by the \f[C]readlink\f[] policy after the mtime and ctime are older than the timeout. .PP -\f[B]WARNING:\f[R] The current implementation has a known issue in which +\f[B]WARNING:\f[] The current implementation has a known issue in which if the file is open and being used when the file is converted to a symlink then the application which has that file open will receive an error when using it. This is unlikely to occur in practice but is something to keep in mind. .PP -\f[B]WARNING:\f[R] Some backup solutions, such as CrashPlan, do not +\f[B]WARNING:\f[] Some backup solutions, such as CrashPlan, do not backup the target of a symlink. If using this feature it will be necessary to point any backup software to the original drives or configure the software to follow symlinks if @@ -564,46 +567,46 @@ One for backup and one for general consumption. .SS nullrw .PP Due to how FUSE works there is an overhead to all requests made to a -FUSE filesystem that wouldn\[cq]t exist for an in kernel one. +FUSE filesystem that wouldn\[aq]t exist for an in kernel one. Meaning that even a simple passthrough will have some slowdown. However, generally the overhead is minimal in comparison to the cost of the underlying I/O. By disabling the underlying I/O we can test the theoretical performance boundaries. .PP -By enabling \f[C]nullrw\f[R] mergerfs will work as it always does -\f[B]except\f[R] that all reads and writes will be no\-ops. +By enabling \f[C]nullrw\f[] mergerfs will work as it always does +\f[B]except\f[] that all reads and writes will be no\-ops. A write will succeed (the size of the write will be returned as if it were successful) but mergerfs does nothing with the data it was given. -Similarly a read will return the size requested but won\[cq]t touch the +Similarly a read will return the size requested but won\[aq]t touch the buffer. .PP See the BENCHMARKING section for suggestions on how to test. .SS xattr .PP -Runtime extended attribute support can be managed via the -\f[C]xattr\f[R] option. +Runtime extended attribute support can be managed via the \f[C]xattr\f[] +option. By default it will passthrough any xattr calls. Given xattr support is rarely used and can have significant performance implications mergerfs allows it to be disabled at runtime. The performance problems mostly comes when file caching is enabled. -The kernel will send a \f[C]getxattr\f[R] for -\f[C]security.capability\f[R] \f[I]before every single write\f[R]. -It doesn\[cq]t cache the responses to any \f[C]getxattr\f[R]. +The kernel will send a \f[C]getxattr\f[] for +\f[C]security.capability\f[] \f[I]before every single write\f[]. +It doesn\[aq]t cache the responses to any \f[C]getxattr\f[]. This might be addressed in the future but for now mergerfs can really only offer the following workarounds. .PP -\f[C]noattr\f[R] will cause mergerfs to short circuit all xattr calls -and return ENOATTR where appropriate. +\f[C]noattr\f[] will cause mergerfs to short circuit all xattr calls and +return ENOATTR where appropriate. mergerfs still gets all the requests but they will not be forwarded on to the underlying filesystems. The runtime control will still function in this mode. .PP -\f[C]nosys\f[R] will cause mergerfs to return ENOSYS for any xattr call. -The difference with \f[C]noattr\f[R] is that the kernel will cache this +\f[C]nosys\f[] will cause mergerfs to return ENOSYS for any xattr call. +The difference with \f[C]noattr\f[] is that the kernel will cache this fact and itself short circuit future calls. -This is more efficient than \f[C]noattr\f[R] but will cause -mergerfs\[cq] runtime control via the hidden file to stop working. +This is more efficient than \f[C]noattr\f[] but will cause mergerfs\[aq] +runtime control via the hidden file to stop working. .SS nfsopenhack .PP NFS is not fully POSIX compliant and historically certain behaviors, @@ -620,53 +623,53 @@ of the NFS server and clients but it results in a permission error because a normal user is not allowed to open a read\-only file as writable. .PP -Even though it\[cq]s a more niche situation this hack breaks normal -security and behavior and as such is \f[C]off\f[R] by default. -If set to \f[C]git\f[R] it will only perform the hack when the path in -question includes \f[C]/.git/\f[R]. -\f[C]all\f[R] will result it it applying anytime a readonly file which -is empty is opened for writing. +Even though it\[aq]s a more niche situation this hack breaks normal +security and behavior and as such is \f[C]off\f[] by default. +If set to \f[C]git\f[] it will only perform the hack when the path in +question includes \f[C]/.git/\f[]. +\f[C]all\f[] will result it it applying anytime a readonly file which is +empty is opened for writing. .SH FUNCTIONS, CATEGORIES and POLICIES .PP The POSIX filesystem API is made up of a number of functions. -\f[B]creat\f[R], \f[B]stat\f[R], \f[B]chown\f[R], etc. +\f[B]creat\f[], \f[B]stat\f[], \f[B]chown\f[], etc. For ease of configuration in mergerfs most of the core functions are -grouped into 3 categories: \f[B]action\f[R], \f[B]create\f[R], and -\f[B]search\f[R]. +grouped into 3 categories: \f[B]action\f[], \f[B]create\f[], and +\f[B]search\f[]. These functions and categories can be assigned a policy which dictates which branch is chosen when performing that function. .PP -Some functions, listed in the category \f[C]N/A\f[R] below, can not be +Some functions, listed in the category \f[C]N/A\f[] below, can not be assigned the normal policies. These functions work with file handles, rather than file paths, which -were created by \f[C]open\f[R] or \f[C]create\f[R]. +were created by \f[C]open\f[] or \f[C]create\f[]. That said many times the current FUSE kernel driver will not always -provide the file handle when a client calls \f[C]fgetattr\f[R], -\f[C]fchown\f[R], \f[C]fchmod\f[R], \f[C]futimens\f[R], -\f[C]ftruncate\f[R], etc. +provide the file handle when a client calls \f[C]fgetattr\f[], +\f[C]fchown\f[], \f[C]fchmod\f[], \f[C]futimens\f[], \f[C]ftruncate\f[], +etc. This means it will call the regular, path based, versions. -\f[C]readdir\f[R] has no real need for a policy given the purpose is +\f[C]readdir\f[] has no real need for a policy given the purpose is merely to return a list of entries in a directory. -\f[C]statfs\f[R]\[cq]s behavior can be modified via other options. +\f[C]statfs\f[]\[aq]s behavior can be modified via other options. .PP -When using policies which are based on a branch\[cq]s available space +When using policies which are based on a branch\[aq]s available space the base path provided is used. Not the full path to the file in question. -Meaning that mounts in the branch won\[cq]t be considered in the space +Meaning that mounts in the branch won\[aq]t be considered in the space calculations. -The reason is that it doesn\[cq]t really work for non\-path preserving +The reason is that it doesn\[aq]t really work for non\-path preserving policies and can lead to non\-obvious behaviors. .PP NOTE: While any policy can be assigned to a function or category though some may not be very useful in practice. -For instance: \f[B]rand\f[R] (random) may be useful for file creation -(create) but could lead to very odd behavior if used for \f[C]chmod\f[R] +For instance: \f[B]rand\f[] (random) may be useful for file creation +(create) but could lead to very odd behavior if used for \f[C]chmod\f[] if there were more than one copy of the file. .SS Functions and their Category classifications .PP .TS tab(@); -lw(7.4n) lw(62.6n). +lw(7.9n) lw(62.1n). T{ Category T}@T{ @@ -699,54 +702,52 @@ T} .TE .PP In cases where something may be searched for (such as a path to clone) -\f[B]getattr\f[R] will usually be used. +\f[B]getattr\f[] will usually be used. .SS Policies .PP A policy is the algorithm used to choose a branch or branches for a function to work on. Think of them as ways to filter and sort branches. .PP -Any function in the \f[C]create\f[R] category will clone the relative +Any function in the \f[C]create\f[] category will clone the relative path if needed. -Some other functions (\f[C]rename\f[R],\f[C]link\f[R],\f[C]ioctl\f[R]) -have special requirements or behaviors which you can read more about -below. +Some other functions (\f[C]rename\f[],\f[C]link\f[],\f[C]ioctl\f[]) have +special requirements or behaviors which you can read more about below. .SS Filtering .PP Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting the branches. -Filters include \f[B]minfreespace\f[R], whether or not a branch is +Filters include \f[B]minfreespace\f[], whether or not a branch is mounted read\-only, and the branch tagging (RO,NC,RW). These filters are applied across all policies unless otherwise noted. .IP \[bu] 2 -No \f[B]search\f[R] function policies filter. +No \f[B]search\f[] function policies filter. .IP \[bu] 2 -All \f[B]action\f[R] function policies filter out branches which are -mounted \f[B]read\-only\f[R] or tagged as \f[B]RO (read\-only)\f[R]. +All \f[B]action\f[] function policies filter out branches which are +mounted \f[B]read\-only\f[] or tagged as \f[B]RO (read\-only)\f[]. .IP \[bu] 2 -All \f[B]create\f[R] function policies filter out branches which are -mounted \f[B]read\-only\f[R], tagged \f[B]RO (read\-only)\f[R] or -\f[B]NC (no create)\f[R], or has available space less than -\f[C]minfreespace\f[R]. +All \f[B]create\f[] function policies filter out branches which are +mounted \f[B]read\-only\f[], tagged \f[B]RO (read\-only)\f[] or \f[B]NC +(no create)\f[], or has available space less than \f[C]minfreespace\f[]. .PP Policies may have their own additional filtering such as those that require existing paths to be present. .PP If all branches are filtered an error will be returned. -Typically \f[B]EROFS\f[R] (read\-only filesystem) or \f[B]ENOSPC\f[R] -(no space left on device) depending on the most recent reason for -filtering a branch. -\f[B]ENOENT\f[R] will be returned if no elegible branch is found. +Typically \f[B]EROFS\f[] (read\-only filesystem) or \f[B]ENOSPC\f[] (no +space left on device) depending on the most recent reason for filtering +a branch. +\f[B]ENOENT\f[] will be returned if no elegible branch is found. .SS Path Preservation .PP Policies, as described below, are of two basic types. -\f[C]path preserving\f[R] and \f[C]non\-path preserving\f[R]. +\f[C]path\ preserving\f[] and \f[C]non\-path\ preserving\f[]. .PP -All policies which start with \f[C]ep\f[R] (\f[B]epff\f[R], -\f[B]eplfs\f[R], \f[B]eplus\f[R], \f[B]epmfs\f[R], \f[B]eprand\f[R]) are -\f[C]path preserving\f[R]. -\f[C]ep\f[R] stands for \f[C]existing path\f[R]. +All policies which start with \f[C]ep\f[] (\f[B]epff\f[], +\f[B]eplfs\f[], \f[B]eplus\f[], \f[B]epmfs\f[], \f[B]eprand\f[]) are +\f[C]path\ preserving\f[]. +\f[C]ep\f[] stands for \f[C]existing\ path\f[]. .PP A path preserving policy will only consider drives where the relative path being accessed already exists. @@ -754,15 +755,15 @@ path being accessed already exists. When using non\-path preserving policies paths will be cloned to target drives as necessary. .PP -With the \f[C]msp\f[R] or \f[C]most shared path\f[R] policies they are -defined as \f[C]path preserving\f[R] for the purpose of controlling -\f[C]link\f[R] and \f[C]rename\f[R]\[cq]s behaviors since -\f[C]ignorepponrename\f[R] is available to disable that behavior. +With the \f[C]msp\f[] or \f[C]most\ shared\ path\f[] policies they are +defined as \f[C]path\ preserving\f[] for the purpose of controlling +\f[C]link\f[] and \f[C]rename\f[]\[aq]s behaviors since +\f[C]ignorepponrename\f[] is available to disable that behavior. In mergerfs v3.0 the path preserving behavior of rename and link will likely be separated from the policy all together. .SS Policy descriptions .PP -A policy\[cq]s behavior differs, as mentioned above, based on the +A policy\[aq]s behavior differs, as mentioned above, based on the function it is used with. Sometimes it really might not make sense to even offer certain policies because they are literally the same as others but it makes things a bit @@ -771,7 +772,7 @@ In mergerfs 3.0 this might change. .PP .TS tab(@); -lw(16.2n) lw(53.8n). +lw(16.6n) lw(53.4n). T{ Policy T}@T{ @@ -781,22 +782,22 @@ _ T{ all T}@T{ -Search: Same as \f[B]epall\f[R]. -Action: Same as \f[B]epall\f[R]. -Create: for \f[B]mkdir\f[R], \f[B]mknod\f[R], and \f[B]symlink\f[R] it -will apply to all branches. -\f[B]create\f[R] works like \f[B]ff\f[R]. +Search: Same as \f[B]epall\f[]. +Action: Same as \f[B]epall\f[]. +Create: for \f[B]mkdir\f[], \f[B]mknod\f[], and \f[B]symlink\f[] it will +apply to all branches. +\f[B]create\f[] works like \f[B]ff\f[]. T} T{ epall (existing path, all) T}@T{ -Search: Same as \f[B]epff\f[R] (but more expensive because it -doesn\[cq]t stop after finding a valid branch). +Search: Same as \f[B]epff\f[] (but more expensive because it doesn\[aq]t +stop after finding a valid branch). Action: apply to all found. -Create: for \f[B]mkdir\f[R], \f[B]mknod\f[R], and \f[B]symlink\f[R] it -will apply to all found. -\f[B]create\f[R] works like \f[B]epff\f[R] (but more expensive because -it doesn\[cq]t stop after finding a valid branch). +Create: for \f[B]mkdir\f[], \f[B]mknod\f[], and \f[B]symlink\f[] it will +apply to all found. +\f[B]create\f[] works like \f[B]epff\f[] (but more expensive because it +doesn\[aq]t stop after finding a valid branch). T} T{ epff (existing path, first found) @@ -825,79 +826,79 @@ T} T{ eppfrd (existing path, percentage free random distribution) T}@T{ -Like \f[B]pfrd\f[R] but limited to existing paths. +Like \f[B]pfrd\f[] but limited to existing paths. T} T{ eprand (existing path, random) T}@T{ -Calls \f[B]epall\f[R] and then randomizes. +Calls \f[B]epall\f[] and then randomizes. Returns 1. T} T{ erofs T}@T{ -Exclusively return \f[B]\-1\f[R] with \f[B]errno\f[R] set to -\f[B]EROFS\f[R] (read\-only filesystem). +Exclusively return \f[B]\-1\f[] with \f[B]errno\f[] set to +\f[B]EROFS\f[] (read\-only filesystem). T} T{ ff (first found) T}@T{ -Search: Same as \f[B]epff\f[R]. -Action: Same as \f[B]epff\f[R]. +Search: Same as \f[B]epff\f[]. +Action: Same as \f[B]epff\f[]. Create: Given the order of the drives, as defined at mount time or configured at runtime, act on the first one found. T} T{ lfs (least free space) T}@T{ -Search: Same as \f[B]eplfs\f[R]. -Action: Same as \f[B]eplfs\f[R]. +Search: Same as \f[B]eplfs\f[]. +Action: Same as \f[B]eplfs\f[]. Create: Pick the drive with the least available free space. T} T{ lus (least used space) T}@T{ -Search: Same as \f[B]eplus\f[R]. -Action: Same as \f[B]eplus\f[R]. +Search: Same as \f[B]eplus\f[]. +Action: Same as \f[B]eplus\f[]. Create: Pick the drive with the least used space. T} T{ mfs (most free space) T}@T{ -Search: Same as \f[B]epmfs\f[R]. -Action: Same as \f[B]epmfs\f[R]. +Search: Same as \f[B]epmfs\f[]. +Action: Same as \f[B]epmfs\f[]. Create: Pick the drive with the most available free space. T} T{ msplfs (most shared path, least free space) T}@T{ -Search: Same as \f[B]eplfs\f[R]. -Action: Same as \f[B]eplfs\f[R]. -Create: like \f[B]eplfs\f[R] but walk back the path if it fails to find -a branch at that level. +Search: Same as \f[B]eplfs\f[]. +Action: Same as \f[B]eplfs\f[]. +Create: like \f[B]eplfs\f[] but walk back the path if it fails to find a +branch at that level. T} T{ msplus (most shared path, least used space) T}@T{ -Search: Same as \f[B]eplus\f[R]. -Action: Same as \f[B]eplus\f[R]. -Create: like \f[B]eplus\f[R] but walk back the path if it fails to find -a branch at that level. +Search: Same as \f[B]eplus\f[]. +Action: Same as \f[B]eplus\f[]. +Create: like \f[B]eplus\f[] but walk back the path if it fails to find a +branch at that level. T} T{ mspmfs (most shared path, most free space) T}@T{ -Search: Same as \f[B]epmfs\f[R]. -Action: Same as \f[B]epmfs\f[R]. -Create: like \f[B]epmfs\f[R] but walk back the path if it fails to find -a branch at that level. +Search: Same as \f[B]epmfs\f[]. +Action: Same as \f[B]epmfs\f[]. +Create: like \f[B]epmfs\f[] but walk back the path if it fails to find a +branch at that level. T} T{ msppfrd (most shared path, percentage free random distribution) T}@T{ -Search: Same as \f[B]eppfrd\f[R]. -Action: Same as \f[B]eppfrd\f[R]. -Create: Like \f[B]eppfrd\f[R] but will walk back the path if it fails to +Search: Same as \f[B]eppfrd\f[]. +Action: Same as \f[B]eppfrd\f[]. +Create: Like \f[B]eppfrd\f[] but will walk back the path if it fails to find a branch at that level. T} T{ @@ -908,27 +909,27 @@ T} T{ pfrd (percentage free random distribution) T}@T{ -Search: Same as \f[B]eppfrd\f[R]. -Action: Same as \f[B]eppfrd\f[R]. +Search: Same as \f[B]eppfrd\f[]. +Action: Same as \f[B]eppfrd\f[]. Create: Chooses a branch at random with the likelihood of selection -based on a branch\[cq]s available space relative to the total. +based on a branch\[aq]s available space relative to the total. T} T{ rand (random) T}@T{ -Calls \f[B]all\f[R] and then randomizes. +Calls \f[B]all\f[] and then randomizes. Returns 1. T} .TE .PP -\f[B]NOTE:\f[R] If you are using an underlying filesystem that reserves +\f[B]NOTE:\f[] If you are using an underlying filesystem that reserves blocks such as ext2, ext3, or ext4 be aware that mergerfs respects the -reservation by using \f[C]f_bavail\f[R] (number of free blocks for -unprivileged users) rather than \f[C]f_bfree\f[R] (number of free -blocks) in policy calculations. -\f[B]df\f[R] does NOT use \f[C]f_bavail\f[R], it uses \f[C]f_bfree\f[R], -so direct comparisons between \f[B]df\f[R] output and mergerfs\[cq] -policies is not appropriate. +reservation by using \f[C]f_bavail\f[] (number of free blocks for +unprivileged users) rather than \f[C]f_bfree\f[] (number of free blocks) +in policy calculations. +\f[B]df\f[] does NOT use \f[C]f_bavail\f[], it uses \f[C]f_bfree\f[], so +direct comparisons between \f[B]df\f[] output and mergerfs\[aq] policies +is not appropriate. .SS Defaults .PP .TS @@ -958,27 +959,26 @@ T} .TE .SS ioctl .PP -When \f[C]ioctl\f[R] is used with an open file then it will use the file -handle which was created at the original \f[C]open\f[R] call. -However, when using \f[C]ioctl\f[R] with a directory mergerfs will use -the \f[C]open\f[R] policy to find the directory to act on. +When \f[C]ioctl\f[] is used with an open file then it will use the file +handle which was created at the original \f[C]open\f[] call. +However, when using \f[C]ioctl\f[] with a directory mergerfs will use +the \f[C]open\f[] policy to find the directory to act on. .SS rename & link .PP -\f[B]NOTE:\f[R] If you\[cq]re receiving errors from software when files +\f[B]NOTE:\f[] If you\[aq]re receiving errors from software when files are moved / renamed / linked then you should consider changing the -create policy to one which is \f[B]not\f[R] path preserving, enabling -\f[C]ignorepponrename\f[R], or contacting the author of the offending -software and requesting that \f[C]EXDEV\f[R] (cross device / improper +create policy to one which is \f[B]not\f[] path preserving, enabling +\f[C]ignorepponrename\f[], or contacting the author of the offending +software and requesting that \f[C]EXDEV\f[] (cross device / improper link) be properly handled. .PP -\f[C]rename\f[R] and \f[C]link\f[R] are tricky functions in a union +\f[C]rename\f[] and \f[C]link\f[] are tricky functions in a union filesystem. -\f[C]rename\f[R] only works within a single filesystem or device. -If a rename can\[cq]t be done atomically due to the source and +\f[C]rename\f[] only works within a single filesystem or device. +If a rename can\[aq]t be done atomically due to the source and destination paths existing on different mount points it will return -\f[B]\-1\f[R] with \f[B]errno = EXDEV\f[R] (cross device / improper -link). -So if a \f[C]rename\f[R]\[cq]s source and target are on different drives +\f[B]\-1\f[] with \f[B]errno = EXDEV\f[] (cross device / improper link). +So if a \f[C]rename\f[]\[aq]s source and target are on different drives within the pool it creates an issue. .PP Originally mergerfs would return EXDEV whenever a rename was requested @@ -988,22 +988,21 @@ requirements. However, many applications fail to handle EXDEV at all and treat it as a normal error or otherwise handle it poorly. Such apps include: gvfsd\-fuse v1.20.3 and prior, Finder / CIFS/SMB -client in Apple OSX 10.9+, NZBGet, Samba\[cq]s recycling bin feature. +client in Apple OSX 10.9+, NZBGet, Samba\[aq]s recycling bin feature. .PP As a result a compromise was made in order to get most software to work -while still obeying mergerfs\[cq] policies. +while still obeying mergerfs\[aq] policies. Below is the basic logic. .IP \[bu] 2 -If using a \f[B]create\f[R] policy which tries to preserve directory +If using a \f[B]create\f[] policy which tries to preserve directory paths (epff,eplfs,eplus,epmfs) -.RS 2 .IP \[bu] 2 -Using the \f[B]rename\f[R] policy get the list of files to rename +Using the \f[B]rename\f[] policy get the list of files to rename .IP \[bu] 2 For each file attempt rename: .RS 2 .IP \[bu] 2 -If failure with ENOENT (no such file or directory) run \f[B]create\f[R] +If failure with ENOENT (no such file or directory) run \f[B]create\f[] policy .IP \[bu] 2 If create policy returns the same drive as currently evaluating then @@ -1012,10 +1011,10 @@ clone the path Re\-attempt rename .RE .IP \[bu] 2 -If \f[B]any\f[R] of the renames succeed the higher level rename is +If \f[B]any\f[] of the renames succeed the higher level rename is considered a success .IP \[bu] 2 -If \f[B]no\f[R] renames succeed the first error encountered will be +If \f[B]no\f[] renames succeed the first error encountered will be returned .IP \[bu] 2 On success: @@ -1025,32 +1024,28 @@ Remove the target from all drives with no source file .IP \[bu] 2 Remove the source from all drives which failed to rename .RE -.RE .IP \[bu] 2 -If using a \f[B]create\f[R] policy which does \f[B]not\f[R] try to +If using a \f[B]create\f[] policy which does \f[B]not\f[] try to preserve directory paths -.RS 2 .IP \[bu] 2 -Using the \f[B]rename\f[R] policy get the list of files to rename +Using the \f[B]rename\f[] policy get the list of files to rename .IP \[bu] 2 -Using the \f[B]getattr\f[R] policy get the target path +Using the \f[B]getattr\f[] policy get the target path .IP \[bu] 2 For each file attempt rename: .RS 2 .IP \[bu] 2 If the source drive != target drive: -.RS 2 .IP \[bu] 2 Clone target path from target drive to source drive -.RE .IP \[bu] 2 Rename .RE .IP \[bu] 2 -If \f[B]any\f[R] of the renames succeed the higher level rename is +If \f[B]any\f[] of the renames succeed the higher level rename is considered a success .IP \[bu] 2 -If \f[B]no\f[R] renames succeed the first error encountered will be +If \f[B]no\f[] renames succeed the first error encountered will be returned .IP \[bu] 2 On success: @@ -1060,14 +1055,13 @@ Remove the target from all drives with no source file .IP \[bu] 2 Remove the source from all drives which failed to rename .RE -.RE .PP The the removals are subject to normal entitlement checks. .PP The above behavior will help minimize the likelihood of EXDEV being returned but it will still be possible. .PP -\f[B]link\f[R] uses the same strategy but without the removals. +\f[B]link\f[] uses the same strategy but without the removals. .SS readdir .PP readdir (http://linux.die.net/man/3/readdir) is different from all other @@ -1076,11 +1070,11 @@ While it could have its own set of policies to tweak its behavior at this time it provides a simple union of files and directories found. Remember that any action or information queried about these files and directories come from the respective function. -For instance: an \f[B]ls\f[R] is a \f[B]readdir\f[R] and for each -file/directory returned \f[B]getattr\f[R] is called. -Meaning the policy of \f[B]getattr\f[R] is responsible for choosing the +For instance: an \f[B]ls\f[] is a \f[B]readdir\f[] and for each +file/directory returned \f[B]getattr\f[] is called. +Meaning the policy of \f[B]getattr\f[] is responsible for choosing the file/directory which is the source of the metadata you see in an -\f[B]ls\f[R]. +\f[B]ls\f[]. .SS statfs / statvfs .PP statvfs (http://linux.die.net/man/2/statvfs) normalizes the source @@ -1091,10 +1085,10 @@ Total, used, and free. The sources however are dedupped based on the drive so multiple sources on the same drive will not result in double counting its space. Filesystems mounted further down the tree of the branch will not be -included when checking the mount\[cq]s stats. +included when checking the mount\[aq]s stats. .PP -The options \f[C]statfs\f[R] and \f[C]statfs_ignore\f[R] can be used to -modify \f[C]statfs\f[R] behavior. +The options \f[C]statfs\f[] and \f[C]statfs_ignore\f[] can be used to +modify \f[C]statfs\f[] behavior. .SH ERROR HANDLING .PP POSIX filesystem functions offer a single return code meaning that there @@ -1131,47 +1125,47 @@ For search functions there is always a single thing acted on and as such whatever return value that comes from the single function call is returned. .PP -For create functions \f[C]mkdir\f[R], \f[C]mknod\f[R], and -\f[C]symlink\f[R] which don\[cq]t return a file descriptor and therefore -can have \f[C]all\f[R] or \f[C]epall\f[R] policies it will return -success if any of the calls succeed and an error otherwise. +For create functions \f[C]mkdir\f[], \f[C]mknod\f[], and +\f[C]symlink\f[] which don\[aq]t return a file descriptor and therefore +can have \f[C]all\f[] or \f[C]epall\f[] policies it will return success +if any of the calls succeed and an error otherwise. .SH BUILD / UPDATE .PP -\f[B]NOTE:\f[R] Prebuilt packages can be found at and recommended for -most users: https://github.com/trapexit/mergerfs/releases -\f[B]NOTE:\f[R] Only tagged releases are supported. -\f[C]master\f[R] and other branches should be considered works in +\f[B]NOTE:\f[] Prebuilt packages can be found at and recommended for +most users: https://github.com/trapexit/mergerfs/releases \f[B]NOTE:\f[] +Only tagged releases are supported. +\f[C]master\f[] and other branches should be considered works in progress. .PP First get the code from github (https://github.com/trapexit/mergerfs). .IP .nf \f[C] -$ git clone https://github.com/trapexit/mergerfs.git -$ # or -$ wget https://github.com/trapexit/mergerfs/releases/download//mergerfs\-.tar.gz -\f[R] +$\ git\ clone\ https://github.com/trapexit/mergerfs.git +$\ #\ or +$\ wget\ https://github.com/trapexit/mergerfs/releases/download//mergerfs\-.tar.gz +\f[] .fi .SS Debian / Ubuntu .IP .nf \f[C] -$ cd mergerfs -$ sudo tools/install\-build\-pkgs -$ make deb -$ sudo dpkg \-i ../mergerfs__.deb -\f[R] +$\ cd\ mergerfs +$\ sudo\ tools/install\-build\-pkgs +$\ make\ deb +$\ sudo\ dpkg\ \-i\ ../mergerfs__.deb +\f[] .fi .SS RHEL / CentOS /Fedora .IP .nf \f[C] -$ su \- -# cd mergerfs -# tools/install\-build\-pkgs -# make rpm -# rpm \-i rpmbuild/RPMS//mergerfs\-..rpm -\f[R] +$\ su\ \- +#\ cd\ mergerfs +#\ tools/install\-build\-pkgs +#\ make\ rpm +#\ rpm\ \-i\ rpmbuild/RPMS//mergerfs\-..rpm +\f[] .fi .SS Generically .PP @@ -1179,22 +1173,22 @@ Have git, g++, make, python installed. .IP .nf \f[C] -$ cd mergerfs -$ make -$ sudo make install -\f[R] +$\ cd\ mergerfs +$\ make +$\ sudo\ make\ install +\f[] .fi .SS Build options .IP .nf \f[C] -$ make help -usage: make +$\ make\ help +usage:\ make -make USE_XATTR=0 \- build program without xattrs functionality -make STATIC=1 \- build static binary -make LTO=1 \- build with link time optimization -\f[R] +make\ USE_XATTR=0\ \ \ \ \ \ \-\ build\ program\ without\ xattrs\ functionality +make\ STATIC=1\ \ \ \ \ \ \ \ \ \-\ build\ static\ binary +make\ LTO=1\ \ \ \ \ \ \ \ \ \ \ \ \-\ build\ with\ link\ time\ optimization +\f[] .fi .SH UPGRADE .PP @@ -1203,28 +1197,28 @@ instance. Simply install the new version of mergerfs and follow the instructions below. .PP -Add \f[C]nonempty\f[R] to your mergerfs option list and call mergerfs -again or if using \f[C]/etc/fstab\f[R] call for it to mount again. +Add \f[C]nonempty\f[] to your mergerfs option list and call mergerfs +again or if using \f[C]/etc/fstab\f[] call for it to mount again. Existing open files and such will continue to work fine though they -won\[cq]t see runtime changes since any such change would be the new +won\[aq]t see runtime changes since any such change would be the new mount. If you plan on changing settings with the new mount you should / could apply those before mounting the new version. .IP .nf \f[C] -$ sudo mount /mnt/mergerfs -$ mount | grep mergerfs -media on /mnt/mergerfs type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) -media on /mnt/mergerfs type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) -\f[R] +$\ sudo\ mount\ /mnt/mergerfs +$\ mount\ |\ grep\ mergerfs +media\ on\ /mnt/mergerfs\ type\ fuse.mergerfs\ (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) +media\ on\ /mnt/mergerfs\ type\ fuse.mergerfs\ (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other) +\f[] .fi .PP A problem with this approach is that the underlying instance will continue to run even if the software using it stop or are restarted. -To work around this you can use a \[lq]lazy umount\[rq]. +To work around this you can use a "lazy umount". Before mounting over top the mount point with the new instance of -mergerfs issue: \f[C]umount \-l \f[R]. +mergerfs issue: \f[C]umount\ \-l\ \f[]. .SH RUNTIME CONFIG .SS ioctl .PP @@ -1239,42 +1233,35 @@ the config file. .PP All commands take a 4096 byte char buffer. .IP \[bu] 2 -read keys: get a nul `\[rs]0' delimited list of option keys -.RS 2 +read keys: get a nul \[aq]\[aq] delimited list of option keys .IP \[bu] 2 _IOWR(0xDF,0,char[4096]) = 0xD000DF00 .IP \[bu] 2 on success ioctl return value is the total length -.RE .IP \[bu] 2 read value: get an option value -.RS 2 .IP \[bu] 2 _IOWR(0xDF,1,char[4096]) = 0xD000DF01 .IP \[bu] 2 -the key is passed in via the char buffer as a nul `\[rs]0' terminated +the key is passed in via the char buffer as a nul \[aq]\[aq] terminated string .IP \[bu] 2 on success ioctl return value is the total length -.RE .IP \[bu] 2 write value: set an option value -.RS 2 .IP \[bu] 2 _IOW(0xDF,2,char[4096]) = 0x5000DF02 .IP \[bu] 2 -the key and value is passed in via the char buffer as a nul `\[rs]0' -terminated string in the format of \f[C]key=value\f[R] +the key and value is passed in via the char buffer as a nul \[aq]\[aq] +terminated string in the format of \f[C]key=value\f[] .IP \[bu] 2 on success ioctl return value is 0 -.RE .IP \[bu] 2 file info: get mergerfs metadata info for a file -.RS 2 .IP \[bu] 2 _IOWR(0xDF,3,char[4096]) = 0xD000DF03 .IP \[bu] 2 -the key is passed in via the char buffer as a nul `\[rs]0' terminated +the key is passed in via the char buffer as a nul \[aq]\[aq] terminated string .IP \[bu] 2 on success the ioctl return value is the total length @@ -1290,8 +1277,8 @@ relpath: the relative path of the file from the mount point fullpath: the full path of the underlying file according to the getattr policy .IP \[bu] 2 -allpaths: a NUL `\[rs]0' delimited list of full paths to all files found -.RE +allpaths: a NUL \[aq]\[aq] delimited list of full paths to all files +found .RE .SS .mergerfs pseudo file (deprecated) .PP @@ -1300,31 +1287,32 @@ NOTE: this interface will be removed in mergerfs 3.0 .nf \f[C] /.mergerfs -\f[R] +\f[] .fi .PP There is a pseudo file available at the mount point which allows for the -runtime modification of certain \f[B]mergerfs\f[R] options. -The file will not show up in \f[B]readdir\f[R] but can be -\f[B]stat\f[R]\[cq]ed and manipulated via +runtime modification of certain \f[B]mergerfs\f[] options. +The file will not show up in \f[B]readdir\f[] but can be +\f[B]stat\f[]\[aq]ed and manipulated via {list,get,set}xattrs (http://linux.die.net/man/2/listxattr) calls. .PP -Any changes made at runtime are \f[B]not\f[R] persisted. +Any changes made at runtime are \f[B]not\f[] persisted. If you wish for values to persist they must be included as options wherever you configure the mounting of mergerfs (/etc/fstab). .SS Keys .PP -Use \f[C]xattr \-l /mountpoint/.mergerfs\f[R] to see all supported keys. +Use \f[C]xattr\ \-l\ /mountpoint/.mergerfs\f[] to see all supported +keys. Some are informational and therefore read\-only. -\f[C]setxattr\f[R] will return EINVAL (invalid argument) on read\-only +\f[C]setxattr\f[] will return EINVAL (invalid argument) on read\-only keys. .SS Values .PP Same as the command line. .SS user.mergerfs.branches .PP -\f[B]NOTE:\f[R] formerly \f[C]user.mergerfs.srcmounts\f[R] but said key -is still supported. +\f[B]NOTE:\f[] formerly \f[C]user.mergerfs.srcmounts\f[] but said key is +still supported. .PP Used to query or modify the list of branches. When modifying there are several shortcuts to easy manipulation of the @@ -1371,80 +1359,79 @@ remove last in list T} .TE .PP -\f[C]xattr \-w user.mergerfs.branches + userspace round trips there are kernel side caches for file entries and attributes. -The entry cache limits the \f[C]lookup\f[R] calls to mergerfs which ask +The entry cache limits the \f[C]lookup\f[] calls to mergerfs which ask if a file exists. -The attribute cache limits the need to make \f[C]getattr\f[R] calls to +The attribute cache limits the need to make \f[C]getattr\f[] calls to mergerfs which provide file attributes (mode, size, type, etc.). As with the page cache these should not be used if the underlying filesystems are being manipulated at the same time as it could lead to odd behavior or data corruption. -The options for setting these are \f[C]cache.entry\f[R] and -\f[C]cache.negative_entry\f[R] for the entry cache and -\f[C]cache.attr\f[R] for the attributes cache. -\f[C]cache.negative_entry\f[R] refers to the timeout for negative +The options for setting these are \f[C]cache.entry\f[] and +\f[C]cache.negative_entry\f[] for the entry cache and +\f[C]cache.attr\f[] for the attributes cache. +\f[C]cache.negative_entry\f[] refers to the timeout for negative responses to lookups (non\-existent files). .SS writeback caching .PP -When \f[C]cache.files\f[R] is enabled the default is for it to perform +When \f[C]cache.files\f[] is enabled the default is for it to perform writethrough caching. -This behavior won\[cq]t help improve performance as each write still +This behavior won\[aq]t help improve performance as each write still goes one for one through the filesystem. By enabling the FUSE writeback cache small writes may be aggregated by the kernel and then sent to mergerfs as one larger request. @@ -1567,15 +1549,15 @@ This can greatly improve the throughput for apps which write to files inefficiently. The amount the kernel can aggregate is limited by the size of a FUSE message. -Read the \f[C]fuse_msg_size\f[R] section for more details. +Read the \f[C]fuse_msg_size\f[] section for more details. .PP There is a small side effect as a result of enabling writeback caching. -Underlying files won\[cq]t ever be opened with O_APPEND or O_WRONLY. +Underlying files won\[aq]t ever be opened with O_APPEND or O_WRONLY. The former because the kernel then manages append mode and the latter because the kernel may request file data from mergerfs to populate the write cache. The O_APPEND change means that if a file is changed outside of mergerfs -it could lead to corruption as the kernel won\[cq]t know the end of the +it could lead to corruption as the kernel won\[aq]t know the end of the file has changed. That said any time you use caching you should keep from using the same file outside of mergerfs at the same time. @@ -1588,64 +1570,62 @@ on older kernels, 1M on newer). .PP Policies are run every time a function (with a policy as mentioned above) is called. -These policies can be expensive depending on mergerfs\[cq] setup and +These policies can be expensive depending on mergerfs\[aq] setup and client usage patterns. -Generally we wouldn\[cq]t want to cache policy results because it may +Generally we wouldn\[aq]t want to cache policy results because it may result in stale responses if the underlying drives are used directly. .PP -The \f[C]open\f[R] policy cache will cache the result of an -\f[C]open\f[R] policy for a particular input for \f[C]cache.open\f[R] -seconds or until the file is unlinked. +The \f[C]open\f[] policy cache will cache the result of an \f[C]open\f[] +policy for a particular input for \f[C]cache.open\f[] seconds or until +the file is unlinked. Each file close (release) will randomly chose to clean up the cache of expired entries. .PP This cache is really only useful in cases where you have a large number -of branches and \f[C]open\f[R] is called on the same files repeatedly -(like \f[B]Transmission\f[R] which opens and closes a file on every +of branches and \f[C]open\f[] is called on the same files repeatedly +(like \f[B]Transmission\f[] which opens and closes a file on every read/write presumably to keep file handle usage low). .SS statfs caching .PP -Of the syscalls used by mergerfs in policies the \f[C]statfs\f[R] / -\f[C]statvfs\f[R] call is perhaps the most expensive. -It\[cq]s used to find out the available space of a drive and whether it +Of the syscalls used by mergerfs in policies the \f[C]statfs\f[] / +\f[C]statvfs\f[] call is perhaps the most expensive. +It\[aq]s used to find out the available space of a drive and whether it is mounted read\-only. Depending on the setup and usage pattern these queries can be relatively costly. -When \f[C]cache.statfs\f[R] is enabled all calls to \f[C]statfs\f[R] by -a policy will be cached for the number of seconds its set to. +When \f[C]cache.statfs\f[] is enabled all calls to \f[C]statfs\f[] by a +policy will be cached for the number of seconds its set to. .PP -Example: If the create policy is \f[C]mfs\f[R] and the timeout is 60 -then for that 60 seconds the same drive will be returned as the target -for creates because the available space won\[cq]t be updated for that -time. +Example: If the create policy is \f[C]mfs\f[] and the timeout is 60 then +for that 60 seconds the same drive will be returned as the target for +creates because the available space won\[aq]t be updated for that time. .SS symlink caching .PP As of version 4.20 Linux supports symlink caching. Significant performance increases can be had in workloads which use a lot of symlinks. -Setting \f[C]cache.symlinks=true\f[R] will result in requesting symlink +Setting \f[C]cache.symlinks=true\f[] will result in requesting symlink caching from the kernel only if supported. As a result its safe to enable it on systems prior to 4.20. That said it is disabled by default for now. You can see if caching is enabled by querying the xattr -\f[C]user.mergerfs.cache.symlinks\f[R] but given it must be requested at +\f[C]user.mergerfs.cache.symlinks\f[] but given it must be requested at startup you can not change it at runtime. .SS readdir caching .PP As of version 4.20 Linux supports readdir caching. This can have a significant impact on directory traversal. -Especially when combined with entry (\f[C]cache.entry\f[R]) and -attribute (\f[C]cache.attr\f[R]) caching. -Setting \f[C]cache.readdir=true\f[R] will result in requesting readdir -caching from the kernel on each \f[C]opendir\f[R]. -If the kernel doesn\[cq]t support readdir caching setting the option to -\f[C]true\f[R] has no effect. +Especially when combined with entry (\f[C]cache.entry\f[]) and attribute +(\f[C]cache.attr\f[]) caching. +Setting \f[C]cache.readdir=true\f[] will result in requesting readdir +caching from the kernel on each \f[C]opendir\f[]. +If the kernel doesn\[aq]t support readdir caching setting the option to +\f[C]true\f[] has no effect. This option is configurable at runtime via xattr -\f[C]user.mergerfs.cache.readdir\f[R]. +\f[C]user.mergerfs.cache.readdir\f[]. .SS tiered caching .PP -Some storage technologies support what some call \[lq]tiered\[rq] -caching. +Some storage technologies support what some call "tiered" caching. The placing of usually smaller, faster storage as a transparent cache to larger, slower storage. NVMe, SSD, Optane in front of traditional HDDs for instance. @@ -1656,18 +1636,18 @@ complicate the code. However, there are a few situations where a cache drive could help with a typical mergerfs setup. .IP "1." 3 -Fast network, slow drives, many readers: You\[cq]ve a 10+Gbps network -with many readers and your regular drives can\[cq]t keep up. +Fast network, slow drives, many readers: You\[aq]ve a 10+Gbps network +with many readers and your regular drives can\[aq]t keep up. .IP "2." 3 -Fast network, slow drives, small\[cq]ish bursty writes: You have a +Fast network, slow drives, small\[aq]ish bursty writes: You have a 10+Gbps network and wish to transfer amounts of data less than your cache drive but wish to do so quickly. .PP With #1 its arguable if you should be using mergerfs at all. RAID would probably be the better solution. -If you\[cq]re going to use mergerfs there are other tactics that may +If you\[aq]re going to use mergerfs there are other tactics that may help: spreading the data across drives (see the mergerfs.dup tool) and -setting \f[C]func.open=rand\f[R], using \f[C]symlinkify\f[R], or using +setting \f[C]func.open=rand\f[], using \f[C]symlinkify\f[], or using dm\-cache or a similar technology to add tiered cache to the underlying device. .PP @@ -1678,47 +1658,46 @@ Create 2 mergerfs pools. One which includes just the slow drives and one which has both the fast drives (SSD,NVME,etc.) and slow drives. .IP "2." 3 -The `cache' pool should have the cache drives listed first. +The \[aq]cache\[aq] pool should have the cache drives listed first. .IP "3." 3 -The best \f[C]create\f[R] policies to use for the `cache' pool would -probably be \f[C]ff\f[R], \f[C]epff\f[R], \f[C]lfs\f[R], or -\f[C]eplfs\f[R]. +The best \f[C]create\f[] policies to use for the \[aq]cache\[aq] pool +would probably be \f[C]ff\f[], \f[C]epff\f[], \f[C]lfs\f[], or +\f[C]eplfs\f[]. The latter two under the assumption that the cache drive(s) are far smaller than the backing drives. -If using path preserving policies remember that you\[cq]ll need to +If using path preserving policies remember that you\[aq]ll need to manually create the core directories of those paths you wish to be cached. Be sure the permissions are in sync. -Use \f[C]mergerfs.fsck\f[R] to check / correct them. -You could also tag the slow drives as \f[C]=NC\f[R] though that\[cq]d -mean if the cache drives fill you\[cq]d get \[lq]out of space\[rq] -errors. +Use \f[C]mergerfs.fsck\f[] to check / correct them. +You could also tag the slow drives as \f[C]=NC\f[] though that\[aq]d +mean if the cache drives fill you\[aq]d get "out of space" errors. .IP "4." 3 -Enable \f[C]moveonenospc\f[R] and set \f[C]minfreespace\f[R] +Enable \f[C]moveonenospc\f[] and set \f[C]minfreespace\f[] appropriately. -To make sure there is enough room on the \[lq]slow\[rq] pool you might -want to set \f[C]minfreespace\f[R] to at least as large as the size of -the largest cache drive if not larger. +To make sure there is enough room on the "slow" pool you might want to +set \f[C]minfreespace\f[] to at least as large as the size of the +largest cache drive if not larger. This way in the worst case the whole of the cache drive(s) can be moved to the other drives. .IP "5." 3 Set your programs to use the cache pool. .IP "6." 3 -Save one of the below scripts or create you\[cq]re own. +Save one of the below scripts or create you\[aq]re own. .IP "7." 3 -Use \f[C]cron\f[R] (as root) to schedule the command at whatever +Use \f[C]cron\f[] (as root) to schedule the command at whatever frequency is appropriate for your workflow. .SS time based expiring .PP Move files from cache to backing pool based only on the last time the file was accessed. -Replace \f[C]\-atime\f[R] with \f[C]\-amin\f[R] if you want minutes -rather than days. -May want to use the \f[C]fadvise\f[R] / \f[C]\-\-drop\-cache\f[R] -version of rsync or run rsync with the tool \[lq]nocache\[rq]. +Replace \f[C]\-atime\f[] with \f[C]\-amin\f[] if you want minutes rather +than days. +May want to use the \f[C]fadvise\f[] / \f[C]\-\-drop\-cache\f[] version +of rsync or run rsync with the tool "nocache". .PP -\f[I]NOTE:\f[R] The arguments to these scripts include the cache -\f[B]drive\f[R]. +\f[I]NOTE:\f[] The arguments to these scripts include the cache +\f[B]drive\f[]. Not the pool with the cache drive. You could have data loss if the source is the cache pool. .IP @@ -1726,26 +1705,26 @@ You could have data loss if the source is the cache pool. \f[C] #!/bin/bash -if [ $# != 3 ]; then - echo \[dq]usage: $0 \[dq] - exit 1 +if\ [\ $#\ !=\ 3\ ];\ then +\ \ echo\ "usage:\ $0\ \ \ " +\ \ exit\ 1 fi -CACHE=\[dq]${1}\[dq] -BACKING=\[dq]${2}\[dq] +CACHE="${1}" +BACKING="${2}" N=${3} -find \[dq]${CACHE}\[dq] \-type f \-atime +${N} \-printf \[aq]%P\[rs]n\[aq] | \[rs] - rsync \-\-files\-from=\- \-axqHAXWES \-\-preallocate \-\-remove\-source\-files \[dq]${CACHE}/\[dq] \[dq]${BACKING}/\[dq] -\f[R] +find\ "${CACHE}"\ \-type\ f\ \-atime\ +${N}\ \-printf\ \[aq]%P\\n\[aq]\ |\ \\ +\ \ rsync\ \-\-files\-from=\-\ \-axqHAXWES\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/"\ "${BACKING}/" +\f[] .fi .SS percentage full expiring .PP Move the oldest file from the cache to the backing pool. Continue till below percentage threshold. .PP -\f[I]NOTE:\f[R] The arguments to these scripts include the cache -\f[B]drive\f[R]. +\f[I]NOTE:\f[] The arguments to these scripts include the cache +\f[B]drive\f[]. Not the pool with the cache drive. You could have data loss if the source is the cache pool. .IP @@ -1753,26 +1732,26 @@ You could have data loss if the source is the cache pool. \f[C] #!/bin/bash -if [ $# != 3 ]; then - echo \[dq]usage: $0 \[dq] - exit 1 +if\ [\ $#\ !=\ 3\ ];\ then +\ \ echo\ "usage:\ $0\ \ \ " +\ \ exit\ 1 fi -CACHE=\[dq]${1}\[dq] -BACKING=\[dq]${2}\[dq] +CACHE="${1}" +BACKING="${2}" PERCENTAGE=${3} -set \-o errexit -while [ $(df \-\-output=pcent \[dq]${CACHE}\[dq] | grep \-v Use | cut \-d\[aq]%\[aq] \-f1) \-gt ${PERCENTAGE} ] +set\ \-o\ errexit +while\ [\ $(df\ \-\-output=pcent\ "${CACHE}"\ |\ grep\ \-v\ Use\ |\ cut\ \-d\[aq]%\[aq]\ \-f1)\ \-gt\ ${PERCENTAGE}\ ] do - FILE=$(find \[dq]${CACHE}\[dq] \-type f \-printf \[aq]%A\[at] %P\[rs]n\[aq] | \[rs] - sort | \[rs] - head \-n 1 | \[rs] - cut \-d\[aq] \[aq] \-f2\-) - test \-n \[dq]${FILE}\[dq] - rsync \-axqHAXWESR \-\-preallocate \-\-remove\-source\-files \[dq]${CACHE}/./${FILE}\[dq] \[dq]${BACKING}/\[dq] +\ \ \ \ FILE=$(find\ "${CACHE}"\ \-type\ f\ \-printf\ \[aq]%A\@\ %P\\n\[aq]\ |\ \\ +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ sort\ |\ \\ +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ head\ \-n\ 1\ |\ \\ +\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ cut\ \-d\[aq]\ \[aq]\ \-f2\-) +\ \ \ \ test\ \-n\ "${FILE}" +\ \ \ \ rsync\ \-axqHAXWESR\ \-\-preallocate\ \-\-remove\-source\-files\ "${CACHE}/./${FILE}"\ "${BACKING}/" done -\f[R] +\f[] .fi .SH PERFORMANCE .PP @@ -1781,49 +1760,49 @@ performance is that of the underlying devices. However, given it is a FUSE filesystem working from userspace there is an increase in overhead relative to kernel based solutions. That said the performance can match the theoretical max but it depends -greatly on the system\[cq]s configuration. +greatly on the system\[aq]s configuration. Especially when adding network filesystems into the mix there are many variables which can impact performance. Drive speeds and latency, network speeds and latency, general concurrency, read/write sizes, etc. Unfortunately, given the number of variables it has been difficult to find a single set of settings which provide optimal performance. -If you\[cq]re having performance issues please look over the suggestions +If you\[aq]re having performance issues please look over the suggestions below (including the benchmarking section.) .PP NOTE: be sure to read about these features before changing them to understand what behaviors it may impact .IP \[bu] 2 -enable (or disable) \f[C]splice_move\f[R], \f[C]splice_read\f[R], and -\f[C]splice_write\f[R] +enable (or disable) \f[C]splice_move\f[], \f[C]splice_read\f[], and +\f[C]splice_write\f[] .IP \[bu] 2 -disable \f[C]security_capability\f[R] and/or \f[C]xattr\f[R] +disable \f[C]security_capability\f[] and/or \f[C]xattr\f[] .IP \[bu] 2 -increase cache timeouts \f[C]cache.attr\f[R], \f[C]cache.entry\f[R], -\f[C]cache.negative_entry\f[R] +increase cache timeouts \f[C]cache.attr\f[], \f[C]cache.entry\f[], +\f[C]cache.negative_entry\f[] .IP \[bu] 2 -enable (or disable) page caching (\f[C]cache.files\f[R]) +enable (or disable) page caching (\f[C]cache.files\f[]) .IP \[bu] 2 -enable \f[C]cache.writeback\f[R] +enable \f[C]cache.writeback\f[] .IP \[bu] 2 -enable \f[C]cache.open\f[R] +enable \f[C]cache.open\f[] .IP \[bu] 2 -enable \f[C]cache.statfs\f[R] +enable \f[C]cache.statfs\f[] .IP \[bu] 2 -enable \f[C]cache.symlinks\f[R] +enable \f[C]cache.symlinks\f[] .IP \[bu] 2 -enable \f[C]cache.readdir\f[R] +enable \f[C]cache.readdir\f[] .IP \[bu] 2 change the number of worker threads .IP \[bu] 2 -disable \f[C]posix_acl\f[R] +disable \f[C]posix_acl\f[] .IP \[bu] 2 -disable \f[C]async_read\f[R] +disable \f[C]async_read\f[] .IP \[bu] 2 -test theoretical performance using \f[C]nullrw\f[R] or mounting a ram +test theoretical performance using \f[C]nullrw\f[] or mounting a ram disk .IP \[bu] 2 -use \f[C]symlinkify\f[R] if your data is largely static and read\-only +use \f[C]symlinkify\f[] if your data is largely static and read\-only .IP \[bu] 2 use tiered cache drives .IP \[bu] 2 @@ -1837,40 +1816,40 @@ Filesystems are complicated. They do many things and many of those are interconnected. Additionally, the OS, drivers, hardware, etc. all can impact performance. -Therefore, when benchmarking, it is \f[B]necessary\f[R] that the test +Therefore, when benchmarking, it is \f[B]necessary\f[] that the test focus as narrowly as possible. .PP For most throughput is the key benchmark. -To test throughput \f[C]dd\f[R] is useful but \f[B]must\f[R] be used -with the correct settings in order to ensure the filesystem or device is +To test throughput \f[C]dd\f[] is useful but \f[B]must\f[] be used with +the correct settings in order to ensure the filesystem or device is actually being tested. The OS can and will cache data. Without forcing synchronous reads and writes and/or disabling caching -the values returned will not be representative of the device\[cq]s true +the values returned will not be representative of the device\[aq]s true performance. .PP When benchmarking through mergerfs ensure you only use 1 branch to remove any possibility of the policies complicating the situation. Benchmark the underlying filesystem first and then mount mergerfs over it and test again. -If you\[cq]re experience speeds below your expectation you will need to +If you\[aq]re experience speeds below your expectation you will need to narrow down precisely which component is leading to the slowdown. Preferably test the following in the order listed (but not combined). .IP "1." 3 -Enable \f[C]nullrw\f[R] mode with \f[C]nullrw=true\f[R]. +Enable \f[C]nullrw\f[] mode with \f[C]nullrw=true\f[]. This will effectively make reads and writes no\-ops. Removing the underlying device / filesystem from the equation. This will give us the top theoretical speeds. .IP "2." 3 -Mount mergerfs over \f[C]tmpfs\f[R]. -\f[C]tmpfs\f[R] is a RAM disk. +Mount mergerfs over \f[C]tmpfs\f[]. +\f[C]tmpfs\f[] is a RAM disk. Extremely high speed and very low latency. This is a more realistic best case scenario. -Example: \f[C]mount \-t tmpfs \-o size=2G tmpfs /tmp/tmpfs\f[R] +Example: \f[C]mount\ \-t\ tmpfs\ \-o\ size=2G\ tmpfs\ /tmp/tmpfs\f[] .IP "3." 3 Mount mergerfs over a local drive. NVMe, SSD, HDD, etc. -If you have more than one I\[cq]d suggest testing each of them as drives +If you have more than one I\[aq]d suggest testing each of them as drives and/or controllers (their drivers) could impact performance. .IP "4." 3 Finally, if you intend to use mergerfs with a network filesystem, either @@ -1880,12 +1859,12 @@ each of those alone as above. Once you find the component which has the performance issue you can do further testing with different options to see if they impact performance. -For reads and writes the most relevant would be: \f[C]cache.files\f[R], -\f[C]async_read\f[R], \f[C]splice_move\f[R], \f[C]splice_read\f[R], -\f[C]splice_write\f[R]. +For reads and writes the most relevant would be: \f[C]cache.files\f[], +\f[C]async_read\f[], \f[C]splice_move\f[], \f[C]splice_read\f[], +\f[C]splice_write\f[]. Less likely but relevant when using NFS or with certain filesystems -would be \f[C]security_capability\f[R], \f[C]xattr\f[R], and -\f[C]posix_acl\f[R]. +would be \f[C]security_capability\f[], \f[C]xattr\f[], and +\f[C]posix_acl\f[]. If you find a specific system, drive, filesystem, controller, etc. that performs poorly contact trapexit so he may investigate further. .PP @@ -1893,11 +1872,11 @@ Sometimes the problem is really the application accessing or writing data through mergerfs. Some software use small buffer sizes which can lead to more requests and therefore greater overhead. -You can test this out yourself by replace \f[C]bs=1M\f[R] in the -examples below with \f[C]ibs\f[R] or \f[C]obs\f[R] and using a size of -\f[C]512\f[R] instead of \f[C]1M\f[R]. -In one example test using \f[C]nullrw\f[R] the write speed dropped from -4.9GB/s to 69.7MB/s when moving from \f[C]1M\f[R] to \f[C]512\f[R]. +You can test this out yourself by replace \f[C]bs=1M\f[] in the examples +below with \f[C]ibs\f[] or \f[C]obs\f[] and using a size of \f[C]512\f[] +instead of \f[C]1M\f[]. +In one example test using \f[C]nullrw\f[] the write speed dropped from +4.9GB/s to 69.7MB/s when moving from \f[C]1M\f[] to \f[C]512\f[]. Similar results were had when testing reads. Small writes overhead may be improved by leveraging a write cache but in casual tests little gain was found. @@ -1910,73 +1889,72 @@ Contact trapexit so he may investigate further. .IP .nf \f[C] -$ dd if=/dev/zero of=/mnt/mergerfs/1GB.file bs=1M count=1024 oflag=nocache conv=fdatasync status=progress -\f[R] +$\ dd\ if=/dev/zero\ of=/mnt/mergerfs/1GB.file\ bs=1M\ count=1024\ oflag=nocache\ conv=fdatasync\ status=progress +\f[] .fi .SS read benchmark .IP .nf \f[C] -$ dd if=/mnt/mergerfs/1GB.file of=/dev/null bs=1M count=1024 iflag=nocache conv=fdatasync status=progress -\f[R] +$\ dd\ if=/mnt/mergerfs/1GB.file\ of=/dev/null\ bs=1M\ count=1024\ iflag=nocache\ conv=fdatasync\ status=progress +\f[] .fi .SH TIPS / NOTES .IP \[bu] 2 This document is very literal and thorough. Unless there is a bug things work as described. -If a suspected feature isn\[cq]t mentioned it doesn\[cq]t exist. +If a suspected feature isn\[aq]t mentioned it doesn\[aq]t exist. .IP \[bu] 2 -Ensure you\[cq]re using the latest version. +Ensure you\[aq]re using the latest version. Few distros have the latest version. .IP \[bu] 2 -\f[B]use_ino\f[R] will only work when used with mergerfs 2.18.0 and +\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and above. .IP \[bu] 2 -Run mergerfs as \f[C]root\f[R] (with \f[B]allow_other\f[R]) unless -you\[cq]re merging paths which are owned by the same user otherwise +Run mergerfs as \f[C]root\f[] (with \f[B]allow_other\f[]) unless +you\[aq]re merging paths which are owned by the same user otherwise strange permission issues may arise. .IP \[bu] 2 https://github.com/trapexit/backup\-and\-recovery\-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure. .IP \[bu] 2 -If you don\[cq]t see some directories and files you expect in a merged +If you don\[aq]t see some directories and files you expect in a merged point or policies seem to skip drives be sure the user has permission to all the underlying directories. -Use \f[C]mergerfs.fsck\f[R] to audit the drive for out of sync +Use \f[C]mergerfs.fsck\f[] to audit the drive for out of sync permissions. .IP \[bu] 2 -Do \f[B]not\f[R] use \f[C]cache.files=off\f[R] if you expect -applications (such as rtorrent) to use -mmap (http://linux.die.net/man/2/mmap) files. +Do \f[B]not\f[] use \f[C]cache.files=off\f[] if you expect applications +(such as rtorrent) to use mmap (http://linux.die.net/man/2/mmap) files. Shared mmap is not currently supported in FUSE w/ page caching disabled. -Enabling \f[C]dropcacheonclose\f[R] is recommended when -\f[C]cache.files=partial|full|auto\-full\f[R]. +Enabling \f[C]dropcacheonclose\f[] is recommended when +\f[C]cache.files=partial|full|auto\-full\f[]. .IP \[bu] 2 Kodi (http://kodi.tv), Plex (http://plex.tv), Subsonic (http://subsonic.org), etc. can use directory mtime (http://linux.die.net/man/2/stat) to more efficiently determine whether to scan for new content rather than simply performing a full scan. -If using the default \f[B]getattr\f[R] policy of \f[B]ff\f[R] it\[cq]s +If using the default \f[B]getattr\f[] policy of \f[B]ff\f[] it\[aq]s possible those programs will miss an update on account of it returning -the first directory found\[cq]s \f[B]stat\f[R] info and its a later -directory on another mount which had the \f[B]mtime\f[R] recently +the first directory found\[aq]s \f[B]stat\f[] info and its a later +directory on another mount which had the \f[B]mtime\f[] recently updated. -To fix this you will want to set \f[B]func.getattr=newest\f[R]. -Remember though that this is just \f[B]stat\f[R]. -If the file is later \f[B]open\f[R]\[cq]ed or \f[B]unlink\f[R]\[cq]ed -and the policy is different for those then a completely different file -or directory could be acted on. +To fix this you will want to set \f[B]func.getattr=newest\f[]. +Remember though that this is just \f[B]stat\f[]. +If the file is later \f[B]open\f[]\[aq]ed or \f[B]unlink\f[]\[aq]ed and +the policy is different for those then a completely different file or +directory could be acted on. .IP \[bu] 2 Some policies mixed with some functions may result in strange behaviors. -Not that some of these behaviors and race conditions couldn\[cq]t happen -outside \f[B]mergerfs\f[R] but that they are far more likely to occur on +Not that some of these behaviors and race conditions couldn\[aq]t happen +outside \f[B]mergerfs\f[] but that they are far more likely to occur on account of the attempt to merge together multiple sources of data which could be out of sync due to the different policies. .IP \[bu] 2 -For consistency its generally best to set \f[B]category\f[R] wide -policies rather than individual \f[B]func\f[R]\[cq]s. +For consistency its generally best to set \f[B]category\f[] wide +policies rather than individual \f[B]func\f[]\[aq]s. This will help limit the confusion of tools such as rsync (http://linux.die.net/man/1/rsync). However, the flexibility is there if needed. @@ -1986,9 +1964,9 @@ However, the flexibility is there if needed. .SS directory mtime is not being updated .PP -Remember that the default policy for \f[C]getattr\f[R] is \f[C]ff\f[R]. +Remember that the default policy for \f[C]getattr\f[] is \f[C]ff\f[]. The information for the first directory found will be returned. -If it wasn\[cq]t the directory which had been updated then it will +If it wasn\[aq]t the directory which had been updated then it will appear outdated. .PP The reason this is the default is because any other policy would be more @@ -1997,26 +1975,26 @@ To always return the directory with the most recent mtime or a faked value based on all found would require a scan of all drives. .PP If you always want the directory information from the one with the most -recent mtime then use the \f[C]newest\f[R] policy for \f[C]getattr\f[R]. -.SS `mv /mnt/pool/foo /mnt/disk1/foo' removes `foo' +recent mtime then use the \f[C]newest\f[] policy for \f[C]getattr\f[]. +.SS \[aq]mv /mnt/pool/foo /mnt/disk1/foo\[aq] removes \[aq]foo\[aq] .PP This is not a bug. .PP -Run in verbose mode to better understand what\[cq]s happening: +Run in verbose mode to better understand what\[aq]s happening: .IP .nf \f[C] -$ mv \-v /mnt/pool/foo /mnt/disk1/foo -copied \[aq]/mnt/pool/foo\[aq] \-> \[aq]/mnt/disk1/foo\[aq] -removed \[aq]/mnt/pool/foo\[aq] -$ ls /mnt/pool/foo -ls: cannot access \[aq]/mnt/pool/foo\[aq]: No such file or directory -\f[R] +$\ mv\ \-v\ /mnt/pool/foo\ /mnt/disk1/foo +copied\ \[aq]/mnt/pool/foo\[aq]\ \->\ \[aq]/mnt/disk1/foo\[aq] +removed\ \[aq]/mnt/pool/foo\[aq] +$\ ls\ /mnt/pool/foo +ls:\ cannot\ access\ \[aq]/mnt/pool/foo\[aq]:\ No\ such\ file\ or\ directory +\f[] .fi .PP -\f[C]mv\f[R], when working across devices, is copying the source to +\f[C]mv\f[], when working across devices, is copying the source to target and then removing the source. -Since the source \f[B]is\f[R] the target in this case, depending on the +Since the source \f[B]is\f[] the target in this case, depending on the unlink policy, it will remove the just copied file and other files across the branches. .PP @@ -2025,7 +2003,7 @@ mergerfs.dedup to clean up the old paths or manually remove them from the branches directly. .SS cached memory appears greater than it should be .PP -Use \f[C]cache.files=off\f[R] and/or \f[C]dropcacheonclose=true\f[R]. +Use \f[C]cache.files=off\f[] and/or \f[C]dropcacheonclose=true\f[]. See the section on page caching. .SS NFS clients returning ESTALE / Stale file handle .PP @@ -2041,114 +2019,113 @@ use_ino inodecalc=path\-hash .SS rtorrent fails with ENODEV (No such device) .PP -Be sure to set \f[C]cache.files=partial|full|auto\-full\f[R] or turn off -\f[C]direct_io\f[R]. +Be sure to set \f[C]cache.files=partial|full|auto\-full\f[] or turn off +\f[C]direct_io\f[]. rtorrent and some other applications use mmap (http://linux.die.net/man/2/mmap) to read and write to files and offer no fallback to traditional methods. -FUSE does not currently support mmap while using \f[C]direct_io\f[R]. -There may be a performance penalty on writes with \f[C]direct_io\f[R] -off as well as the problem of double caching but it\[cq]s the only way -to get such applications to work. +FUSE does not currently support mmap while using \f[C]direct_io\f[]. +There may be a performance penalty on writes with \f[C]direct_io\f[] off +as well as the problem of double caching but it\[aq]s the only way to +get such applications to work. If the performance loss is too high for other apps you can mount mergerfs twice. -Once with \f[C]direct_io\f[R] enabled and one without it. -Be sure to set \f[C]dropcacheonclose=true\f[R] if not using -\f[C]direct_io\f[R]. -.SS Plex doesn\[cq]t work with mergerfs +Once with \f[C]direct_io\f[] enabled and one without it. +Be sure to set \f[C]dropcacheonclose=true\f[] if not using +\f[C]direct_io\f[]. +.SS Plex doesn\[aq]t work with mergerfs .PP It does. -If you\[cq]re trying to put Plex\[cq]s config / metadata / database on -mergerfs you can\[cq]t set \f[C]cache.files=off\f[R] because Plex is +If you\[aq]re trying to put Plex\[aq]s config / metadata / database on +mergerfs you can\[aq]t set \f[C]cache.files=off\f[] because Plex is using sqlite3 with mmap enabled. -Shared mmap is not supported by Linux\[cq]s FUSE implementation when +Shared mmap is not supported by Linux\[aq]s FUSE implementation when page caching is disabled. To fix this place the data elsewhere (preferable) or enable -\f[C]cache.files\f[R] (with \f[C]dropcacheonclose=true\f[R]). +\f[C]cache.files\f[] (with \f[C]dropcacheonclose=true\f[]). Sqlite3 does not need mmap but the developer needs to fall back to standard IO if mmap fails. .PP -If the issue is that scanning doesn\[cq]t seem to pick up media then be -sure to set \f[C]func.getattr=newest\f[R] though generally a full scan +If the issue is that scanning doesn\[aq]t seem to pick up media then be +sure to set \f[C]func.getattr=newest\f[] though generally a full scan will pick up all media anyway. .SS When a program tries to move or rename a file it fails .PP -Please read the section above regarding rename & link. +Please read the section above regarding rename & link (#rename--link). .PP The problem is that many applications do not properly handle -\f[C]EXDEV\f[R] errors which \f[C]rename\f[R] and \f[C]link\f[R] may -return even though they are perfectly valid situations which do not -indicate actual drive or OS errors. +\f[C]EXDEV\f[] errors which \f[C]rename\f[] and \f[C]link\f[] may return +even though they are perfectly valid situations which do not indicate +actual drive or OS errors. The error will only be returned by mergerfs if using a path preserving policy as described in the policy section above. If you do not care about path preservation simply change the mergerfs policy to the non\-path preserving version. -For example: \f[C]\-o category.create=mfs\f[R] +For example: \f[C]\-o\ category.create=mfs\f[] .PP Ideally the offending software would be fixed and it is recommended that -if you run into this problem you contact the software\[cq]s author and -request proper handling of \f[C]EXDEV\f[R] errors. +if you run into this problem you contact the software\[aq]s author and +request proper handling of \f[C]EXDEV\f[] errors. .SS my 32bit software has problems .PP Some software have problems with 64bit inode values. The symptoms can include EOVERFLOW errors when trying to list files. -You can address this by setting \f[C]inodecalc\f[R] to one of the 32bit +You can address this by setting \f[C]inodecalc\f[] to one of the 32bit based algos as described in the relevant section. .SS Samba: Moving files / directories fails .PP Workaround: Copy the file/directory and then remove the original rather than move. .PP -This isn\[cq]t an issue with Samba but some SMB clients. +This isn\[aq]t an issue with Samba but some SMB clients. GVFS\-fuse v1.20.3 and prior (found in Ubuntu 14.04 among others) failed to handle certain error codes correctly. -Particularly \f[B]STATUS_NOT_SAME_DEVICE\f[R] which comes from the -\f[B]EXDEV\f[R] which is returned by \f[B]rename\f[R] when the call is +Particularly \f[B]STATUS_NOT_SAME_DEVICE\f[] which comes from the +\f[B]EXDEV\f[] which is returned by \f[B]rename\f[] when the call is crossing mount points. -When a program gets an \f[B]EXDEV\f[R] it needs to explicitly take an +When a program gets an \f[B]EXDEV\f[] it needs to explicitly take an alternate action to accomplish its goal. -In the case of \f[B]mv\f[R] or similar it tries \f[B]rename\f[R] and on -\f[B]EXDEV\f[R] falls back to a manual copying of data between the two +In the case of \f[B]mv\f[] or similar it tries \f[B]rename\f[] and on +\f[B]EXDEV\f[] falls back to a manual copying of data between the two locations and unlinking the source. -In these older versions of GVFS\-fuse if it received \f[B]EXDEV\f[R] it -would translate that into \f[B]EIO\f[R]. -This would cause \f[B]mv\f[R] or most any application attempting to move +In these older versions of GVFS\-fuse if it received \f[B]EXDEV\f[] it +would translate that into \f[B]EIO\f[]. +This would cause \f[B]mv\f[] or most any application attempting to move files around on that SMB share to fail with a IO error. .PP GVFS\-fuse v1.22.0 (https://bugzilla.gnome.org/show_bug.cgi?id=734568) and above fixed this issue but a large number of systems use the older release. On Ubuntu the version can be checked by issuing -\f[C]apt\-cache showpkg gvfs\-fuse\f[R]. +\f[C]apt\-cache\ showpkg\ gvfs\-fuse\f[]. Most distros released in 2015 seem to have the updated release and will work fine but older systems may not. Upgrading gvfs\-fuse or the distro in general will address the problem. .PP -In Apple\[cq]s MacOSX 10.9 they replaced Samba (client and server) with +In Apple\[aq]s MacOSX 10.9 they replaced Samba (client and server) with their own product. -It appears their new client does not handle \f[B]EXDEV\f[R] either and +It appears their new client does not handle \f[B]EXDEV\f[] either and responds similar to older release of gvfs on Linux. .SS Trashing files occasionally fails .PP This is the same issue as with Samba. -\f[C]rename\f[R] returns \f[C]EXDEV\f[R] (in our case that will really -only happen with path preserving policies like \f[C]epmfs\f[R]) and the -software doesn\[cq]t handle the situation well. +\f[C]rename\f[] returns \f[C]EXDEV\f[] (in our case that will really +only happen with path preserving policies like \f[C]epmfs\f[]) and the +software doesn\[aq]t handle the situation well. This is unfortunately a common failure of software which moves files around. -The standard indicates that an implementation \f[C]MAY\f[R] choose to +The standard indicates that an implementation \f[C]MAY\f[] choose to support non\-user home directory trashing of files (which is a -\f[C]MUST\f[R]). -The implementation \f[C]MAY\f[R] also support \[lq]top directory -trashes\[rq] which many probably do. -.PP -To create a \f[C]$topdir/.Trash\f[R] directory as defined in the -standard use the -mergerfs\-tools (https://github.com/trapexit/mergerfs-tools) tool -\f[C]mergerfs.mktrash\f[R]. +\f[C]MUST\f[]). +The implementation \f[C]MAY\f[] also support "top directory trashes" +which many probably do. +.PP +To create a \f[C]$topdir/.Trash\f[] directory as defined in the standard +use the mergerfs\-tools (https://github.com/trapexit/mergerfs-tools) +tool \f[C]mergerfs.mktrash\f[]. .SS tar: Directory renamed before its status could be extracted .PP -Make sure to use the \f[C]use_ino\f[R] option. +Make sure to use the \f[C]use_ino\f[] option. .SS Supplemental user groups .PP Due to the overhead of @@ -2160,14 +2137,14 @@ particular thread needs to change credentials and will keep that data for the lifetime of the thread. This means that if a user is added to a group it may not be picked up without the restart of mergerfs. -However, since the high level FUSE API\[cq]s (at least the standard -version) thread pool dynamically grows and shrinks it\[cq]s possible +However, since the high level FUSE API\[aq]s (at least the standard +version) thread pool dynamically grows and shrinks it\[aq]s possible that over time a thread will be killed and later a new thread with no cache will start and query the new data. .PP The gid cache uses fixed storage to simplify the design and be compatible with older systems which may not have C++11 compilers. -There is enough storage for 256 users\[cq] supplemental groups. +There is enough storage for 256 users\[aq] supplemental groups. Each user is allowed up to 32 supplemental groups. Linux >= 2.6.3 allows up to 65535 groups per user but most other *nixs allow far less. @@ -2176,18 +2153,18 @@ The system does handle overflow gracefully. If the user has more than 32 supplemental groups only the first 32 will be used. If more than 256 users are using the system when an uncached user is -found it will evict an existing user\[cq]s cache at random. -So long as there aren\[cq]t more than 256 active users this should be +found it will evict an existing user\[aq]s cache at random. +So long as there aren\[aq]t more than 256 active users this should be fine. If either value is too low for your needs you will have to modify -\f[C]gidcache.hpp\f[R] to increase the values. +\f[C]gidcache.hpp\f[] to increase the values. Note that doing so will increase the memory needed by each thread. .PP While not a bug some users have found when using containers that -supplemental groups defined inside the container don\[cq]t work properly +supplemental groups defined inside the container don\[aq]t work properly with regard to permissions. This is expected as mergerfs lives outside the container and therefore -is querying the host\[cq]s group database. +is querying the host\[aq]s group database. There might be a hack to work around this (make mergerfs read the /etc/group file in the container) but it is not yet implemented and would be limited to Linux and the /etc/group DB. @@ -2195,15 +2172,16 @@ Preferably users would mount in the host group file into the containers or use a standard shared user & groups technology like NIS or LDAP. .SS mergerfs or libfuse crashing .PP -First\&... always upgrade to the latest version unless told otherwise. +First... +always upgrade to the latest version unless told otherwise. .PP If using mergerfs below 2.22.0: .PP If suddenly the mergerfs mount point disappears and -\f[C]Transport endpoint is not connected\f[R] is returned when -attempting to perform actions within the mount directory \f[B]and\f[R] -the version of libfuse (use \f[C]mergerfs \-v\f[R] to find the version) -is older than \f[C]2.9.4\f[R] its likely due to a bug in libfuse. +\f[C]Transport\ endpoint\ is\ not\ connected\f[] is returned when +attempting to perform actions within the mount directory \f[B]and\f[] +the version of libfuse (use \f[C]mergerfs\ \-v\f[] to find the version) +is older than \f[C]2.9.4\f[] its likely due to a bug in libfuse. Affected versions of libfuse can be found in Debian Wheezy, Ubuntu Precise and others. .PP @@ -2220,10 +2198,10 @@ First upgrade if possible, check the known bugs section, and contact trapexit. .SS mergerfs appears to be crashing or exiting .PP -There seems to be an issue with Linux version \f[C]4.9.0\f[R] and above +There seems to be an issue with Linux version \f[C]4.9.0\f[] and above in which an invalid message appears to be transmitted to libfuse (used by mergerfs) causing it to exit. -No messages will be printed in any logs as it\[cq]s not a proper crash. +No messages will be printed in any logs as it\[aq]s not a proper crash. Debugging of the issue is still ongoing and can be followed via the fuse\-devel thread (https://sourceforge.net/p/fuse/mailman/message/35662577). @@ -2233,63 +2211,64 @@ Please update. This is only happened to mergerfs versions at or below v2.25.x and will not occur in more recent versions. .SH FAQ -.SS How well does mergerfs scale? Is it \[lq]production ready?\[rq] +.SS How well does mergerfs scale? Is it "production ready?" .PP Users have reported running mergerfs on everything from a Raspberry Pi to dual socket Xeon systems with >20 cores. -I\[cq]m aware of at least a few companies which use mergerfs in +I\[aq]m aware of at least a few companies which use mergerfs in production. Open Media Vault (https://www.openmediavault.org) includes mergerfs as its sole solution for pooling drives. The author of mergerfs had it running for over 300 days managing 16+ drives with reasonably heavy 24/7 read and write usage. -Stopping only after the machine\[cq]s power supply died. +Stopping only after the machine\[aq]s power supply died. .PP Most serious issues (crashes or data corruption) have been due to kernel bugs (https://github.com/trapexit/mergerfs/wiki/Kernel-Issues-&-Bugs). All of which are fixed in stable releases. -.SS Can mergerfs be used with drives which already have data / are in use? +.SS Can mergerfs be used with drives which already have data / are in +use? .PP Yes. -MergerFS is a proxy and does \f[B]NOT\f[R] interfere with the normal -form or function of the drives / mounts / paths it manages. -.PP -MergerFS is \f[B]not\f[R] a traditional filesystem. -MergerFS is \f[B]not\f[R] RAID. -It does \f[B]not\f[R] manipulate the data that passes through it. -It does \f[B]not\f[R] shard data across drives. -It merely shards some \f[B]behavior\f[R] and aggregates others. +MergerFS is a proxy and does \f[B]NOT\f[] interfere with the normal form +or function of the drives / mounts / paths it manages. +.PP +MergerFS is \f[B]not\f[] a traditional filesystem. +MergerFS is \f[B]not\f[] RAID. +It does \f[B]not\f[] manipulate the data that passes through it. +It does \f[B]not\f[] shard data across drives. +It merely shards some \f[B]behavior\f[] and aggregates others. .SS Can mergerfs be removed without affecting the data? .PP -See the previous question\[cq]s answer. +See the previous question\[aq]s answer. .SS What policies should I use? .PP -Unless you\[cq]re doing something more niche the average user is -probably best off using \f[C]mfs\f[R] for \f[C]category.create\f[R]. +Unless you\[aq]re doing something more niche the average user is +probably best off using \f[C]mfs\f[] for \f[C]category.create\f[]. It will spread files out across your branches based on available space. -Use \f[C]mspmfs\f[R] if you want to try to colocate the data a bit more. -You may want to use \f[C]lus\f[R] if you prefer a slightly different +Use \f[C]mspmfs\f[] if you want to try to colocate the data a bit more. +You may want to use \f[C]lus\f[] if you prefer a slightly different distribution of data if you have a mix of smaller and larger drives. -Generally though \f[C]mfs\f[R], \f[C]lus\f[R], or even \f[C]rand\f[R] -are good for the general use case. +Generally though \f[C]mfs\f[], \f[C]lus\f[], or even \f[C]rand\f[] are +good for the general use case. If you are starting with an imbalanced pool you can use the tool -\f[B]mergerfs.balance\f[R] to redistribute files across the pool. +\f[B]mergerfs.balance\f[] to redistribute files across the pool. .PP If you really wish to try to colocate files based on directory you can -set \f[C]func.create\f[R] to \f[C]epmfs\f[R] or similar and -\f[C]func.mkdir\f[R] to \f[C]rand\f[R] or \f[C]eprand\f[R] depending on -if you just want to colocate generally or on specific branches. -Either way the \f[I]need\f[R] to colocate is rare. +set \f[C]func.create\f[] to \f[C]epmfs\f[] or similar and +\f[C]func.mkdir\f[] to \f[C]rand\f[] or \f[C]eprand\f[] depending on if +you just want to colocate generally or on specific branches. +Either way the \f[I]need\f[] to colocate is rare. For instance: if you wish to remove the drive regularly and want the -data to predictably be on that drive or if you don\[cq]t use backup at -all and don\[cq]t wish to replace that data piecemeal. +data to predictably be on that drive or if you don\[aq]t use backup at +all and don\[aq]t wish to replace that data piecemeal. In which case using path preservation can help but will require some manual attention. Colocating after the fact can be accomplished using the -\f[B]mergerfs.consolidate\f[R] tool. -If you don\[cq]t need strict colocation which the \f[C]ep\f[R] policies -provide then you can use the \f[C]msp\f[R] based policies which will -walk back the path till finding a branch that works. +\f[B]mergerfs.consolidate\f[] tool. +If you don\[aq]t need strict colocation which the \f[C]ep\f[] policies +provide then you can use the \f[C]msp\f[] based policies which will walk +back the path till finding a branch that works. .PP Ultimately there is no correct answer. It is a preference or based on some particular need. @@ -2297,34 +2276,34 @@ mergerfs is very easy to test and experiment with. I suggest creating a test setup and experimenting to get a sense of what you want. .PP -The reason \f[C]mfs\f[R] is not the default \f[C]category.create\f[R] +The reason \f[C]mfs\f[] is not the default \f[C]category.create\f[] policy is historical. When/if a 3.X gets released it will be changed to minimize confusion people often have with path preserving policies. .SS What settings should I use? .PP Depends on what features you want. -Generally speaking there are no \[lq]wrong\[rq] settings. +Generally speaking there are no "wrong" settings. All settings are performance or feature related. The best bet is to read over the available options and choose what fits your situation. -If something isn\[cq]t clear from the documentation please reach out and +If something isn\[aq]t clear from the documentation please reach out and the documentation will be improved. .PP That said, for the average person, the following should be fine: .PP -\f[C]\-o use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs\f[R] +\f[C]\-o\ use_ino,cache.files=off,dropcacheonclose=true,allow_other,category.create=mfs\f[] .SS Why are all my files ending up on 1 drive?! .PP Did you start with empty drives? -Did you explicitly configure a \f[C]category.create\f[R] policy? -Are you using an \f[C]existing path\f[R] / \f[C]path preserving\f[R] +Did you explicitly configure a \f[C]category.create\f[] policy? +Are you using an \f[C]existing\ path\f[] / \f[C]path\ preserving\f[] policy? .PP -The default create policy is \f[C]epmfs\f[R]. +The default create policy is \f[C]epmfs\f[]. That is a path preserving algorithm. -With such a policy for \f[C]mkdir\f[R] and \f[C]create\f[R] with a set -of empty drives it will select only 1 drive when the first directory is +With such a policy for \f[C]mkdir\f[] and \f[C]create\f[] with a set of +empty drives it will select only 1 drive when the first directory is created. Anything, files or directories, created in that first directory will be placed on the same branch because it is preserving paths. @@ -2332,24 +2311,24 @@ placed on the same branch because it is preserving paths. This catches a lot of new users off guard but changing the default would break the setup for many existing users. If you do not care about path preservation and wish your files to be -spread across all your drives change to \f[C]mfs\f[R] or similar policy +spread across all your drives change to \f[C]mfs\f[] or similar policy as described above. -If you do want path preservation you\[cq]ll need to perform the manual +If you do want path preservation you\[aq]ll need to perform the manual act of creating paths on the drives you want the data to land on before transferring your data. -Setting \f[C]func.mkdir=epall\f[R] can simplify managing path -preservation for \f[C]create\f[R]. -Or use \f[C]func.mkdir=rand\f[R] if you\[cq]re interested in just +Setting \f[C]func.mkdir=epall\f[] can simplify managing path +preservation for \f[C]create\f[]. +Or use \f[C]func.mkdir=rand\f[] if you\[aq]re interested in just grouping together directory content by drive. .SS Do hardlinks work? .PP Yes. -You need to use \f[C]use_ino\f[R] to support proper reporting of inodes +You need to use \f[C]use_ino\f[] to support proper reporting of inodes but they work regardless. -See also the option \f[C]inodecalc\f[R]. +See also the option \f[C]inodecalc\f[]. .PP What mergerfs does not do is fake hard links across branches. -Read the section \[lq]rename & link\[rq] for how it works. +Read the section "rename & link" for how it works. .PP Remember that hardlinks will NOT work across devices. That includes between the original filesystem and a mergerfs pool, @@ -2361,7 +2340,8 @@ considered different devices. There is no way to link between them. You should mount in the highest directory in the mergerfs pool that includes all the paths you need if you want links to work. -.SS Does mergerfs support CoW / copy\-on\-write / writes to read\-only filesystems? +.SS Does mergerfs support CoW / copy\-on\-write / writes to read\-only +filesystems? .PP Not in the sense of a filesystem like BTRFS or ZFS nor in the overlayfs or aufs sense. @@ -2375,9 +2355,9 @@ file. If you want to write to a read\-only filesystem you should look at overlayfs. You can always include the overlayfs mount into a mergerfs pool. -.SS Why can\[cq]t I see my files / directories? +.SS Why can\[aq]t I see my files / directories? .PP -It\[cq]s almost always a permissions issue. +It\[aq]s almost always a permissions issue. Unlike mhddfs and unionfs\-fuse, which runs as root and attempts to access content as such, mergerfs always changes its credentials to that of the caller. @@ -2391,12 +2371,12 @@ Whenever you run into a split permission issue (seeing some but not all files) try using mergerfs.fsck (https://github.com/trapexit/mergerfs-tools) tool to check for and fix the mismatch. -If you aren\[cq]t seeing anything at all be sure that the basic +If you aren\[aq]t seeing anything at all be sure that the basic permissions are correct. The user and group values are correct and that directories have their executable bit set. -A common mistake by users new to Linux is to \f[C]chmod \-R 644\f[R] -when they should have \f[C]chmod \-R u=rwX,go=rX\f[R]. +A common mistake by users new to Linux is to \f[C]chmod\ \-R\ 644\f[] +when they should have \f[C]chmod\ \-R\ u=rwX,go=rX\f[]. .PP If using a network filesystem such as NFS, SMB, CIFS (Samba) be sure to pay close attention to anything regarding permissioning and users. @@ -2404,27 +2384,28 @@ Root squashing and user translation for instance has bitten a few mergerfs users. Some of these also affect the use of mergerfs from container platforms such as Docker. -.SS Is my OS\[cq]s libfuse needed for mergerfs to work? -.PP -No.\ Normally \f[C]mount.fuse\f[R] is needed to get mergerfs (or any -FUSE filesystem to mount using the \f[C]mount\f[R] command but in -vendoring the libfuse library the \f[C]mount.fuse\f[R] app has been -renamed to \f[C]mount.mergerfs\f[R] meaning the filesystem type in -\f[C]fstab\f[R] can simply be \f[C]mergerfs\f[R]. +.SS Is my OS\[aq]s libfuse needed for mergerfs to work? +.PP +No. +Normally \f[C]mount.fuse\f[] is needed to get mergerfs (or any FUSE +filesystem to mount using the \f[C]mount\f[] command but in vendoring +the libfuse library the \f[C]mount.fuse\f[] app has been renamed to +\f[C]mount.mergerfs\f[] meaning the filesystem type in \f[C]fstab\f[] +can simply be \f[C]mergerfs\f[]. That said there should be no harm in having it installed and continuing -to using \f[C]fuse.mergerfs\f[R] as the type in \f[C]/etc/fstab\f[R]. +to using \f[C]fuse.mergerfs\f[] as the type in \f[C]/etc/fstab\f[]. .PP -If \f[C]mergerfs\f[R] doesn\[cq]t work as a type it could be due to how -the \f[C]mount.mergerfs\f[R] tool was installed. -Must be in \f[C]/sbin/\f[R] with proper permissions. +If \f[C]mergerfs\f[] doesn\[aq]t work as a type it could be due to how +the \f[C]mount.mergerfs\f[] tool was installed. +Must be in \f[C]/sbin/\f[] with proper permissions. .SS Why was libfuse embedded into mergerfs? .IP "1." 3 A significant number of users use mergerfs on distros with old versions of libfuse which have serious bugs. -Requiring updated versions of libfuse on those distros isn\[cq]t +Requiring updated versions of libfuse on those distros isn\[aq]t practical (no package offered, user inexperience, etc.). The only practical way to provide a stable runtime on those systems was -to \[lq]vendor\[rq] / embed the library into the project. +to "vendor" / embed the library into the project. .IP "2." 3 mergerfs was written to use the high level API. There are a number of limitations in the HLAPI that make certain @@ -2441,21 +2422,21 @@ Longer term the plan is to rewrite mergerfs to use the low level API. .PP See above first. .PP -If/when mergerfs is rewritten to use the low\-level API then it\[cq]ll +If/when mergerfs is rewritten to use the low\-level API then it\[aq]ll be plausible to support system libfuse but till then its simply too much work to manage the differences across the versions. .SS Why use mergerfs over mhddfs? .PP mhddfs is no longer maintained and has some known stability and security issues (see below). -MergerFS provides a superset of mhddfs\[cq] features and should offer +MergerFS provides a superset of mhddfs\[aq] features and should offer the same or maybe better performance. .PP Below is an example of mhddfs and mergerfs setup to work similarly. .PP -\f[C]mhddfs \-o mlimit=4G,allow_other /mnt/drive1,/mnt/drive2 /mnt/pool\f[R] +\f[C]mhddfs\ \-o\ mlimit=4G,allow_other\ /mnt/drive1,/mnt/drive2\ /mnt/pool\f[] .PP -\f[C]mergerfs \-o minfreespace=4G,allow_other,category.create=ff /mnt/drive1:/mnt/drive2 /mnt/pool\f[R] +\f[C]mergerfs\ \-o\ minfreespace=4G,allow_other,category.create=ff\ /mnt/drive1:/mnt/drive2\ /mnt/pool\f[] .SS Why use mergerfs over aufs? .PP aufs is mostly abandoned and no longer available in many distros. @@ -2468,13 +2449,14 @@ features which aufs and overlayfs have. .PP UnionFS is more like aufs than mergerfs in that it offers overlay / CoW features. -If you\[cq]re just looking to create a union of drives and want +If you\[aq]re just looking to create a union of drives and want flexibility in file/directory placement then mergerfs offers that whereas unionfs is more for overlaying RW filesystems over RO ones. .SS Why use mergerfs over overlayfs? .PP Same reasons as with unionfs. -.SS Why use mergerfs over LVM/ZFS/BTRFS/RAID0 drive concatenation / striping? +.SS Why use mergerfs over LVM/ZFS/BTRFS/RAID0 drive concatenation / +striping? .PP With simple JBOD / drive concatenation / stripping / RAID0 a single drive failure will result in full pool failure. @@ -2490,7 +2472,8 @@ without the single point of failure. MergerFS is not intended to be a replacement for ZFS. MergerFS is intended to provide flexible pooling of arbitrary drives (local or remote), of arbitrary sizes, and arbitrary filesystems. -For \f[C]write once, read many\f[R] usecases such as bulk media storage. +For \f[C]write\ once,\ read\ many\f[] usecases such as bulk media +storage. Where data integrity and backup is managed in other ways. In that situation ZFS can introduce a number of costs and limitations as described @@ -2506,14 +2489,14 @@ flexibility offered by mergerfs and for some the fact it is free and open source is important. .PP There are a number of UnRAID users who use mergerfs as well though -I\[cq]m not entirely familiar with the use case. +I\[aq]m not entirely familiar with the use case. .SS What should mergerfs NOT be used for? .IP \[bu] 2 databases: Even if the database stored data in separate files (mergerfs -wouldn\[cq]t offer much otherwise) the higher latency of the indirection +wouldn\[aq]t offer much otherwise) the higher latency of the indirection will kill performance. If it is a lightly used SQLITE database then it may be fine but -you\[cq]ll need to test. +you\[aq]ll need to test. .IP \[bu] 2 VM images: For the same reasons as databases. VM images are accessed very aggressively and mergerfs will introduce too @@ -2524,60 +2507,61 @@ If you need that kind of device performance aggregation or high availability you should stick with RAID. .SS Can drives be written to directly? Outside of mergerfs while pooled? .PP -Yes, however it\[cq]s not recommended to use the same file from within +Yes, however it\[aq]s not recommended to use the same file from within the pool and from without at the same time (particularly writing). Especially if using caching of any kind (cache.files, cache.entry, cache.attr, cache.negative_entry, cache.symlinks, cache.readdir, etc.) as there could be a conflict between cached data and not. -.SS Why do I get an \[lq]out of space\[rq] / \[lq]no space left on device\[rq] / ENOSPC error even though there appears to be lots of space available? +.SS Why do I get an "out of space" / "no space left on device" / ENOSPC +error even though there appears to be lots of space available? .PP -First make sure you\[cq]ve read the sections above about policies, path -preservation, branch filtering, and the options \f[B]minfreespace\f[R], -\f[B]moveonenospc\f[R], \f[B]statfs\f[R], and \f[B]statfs_ignore\f[R]. +First make sure you\[aq]ve read the sections above about policies, path +preservation, branch filtering, and the options \f[B]minfreespace\f[], +\f[B]moveonenospc\f[], \f[B]statfs\f[], and \f[B]statfs_ignore\f[]. .PP mergerfs is simply presenting a union of the content within multiple branches. The reported free space is an aggregate of space available within the -pool (behavior modified by \f[B]statfs\f[R] and -\f[B]statfs_ignore\f[R]). +pool (behavior modified by \f[B]statfs\f[] and \f[B]statfs_ignore\f[]). It does not represent a contiguous space. In the same way that read\-only filesystems, those with quotas, or reserved space report the full theoretical space available. .PP Due to path preservation, branch tagging, read\-only status, and -\f[B]minfreespace\f[R] settings it is perfectly valid that -\f[C]ENOSPC\f[R] / \[lq]out of space\[rq] / \[lq]no space left on -device\[rq] be returned. +\f[B]minfreespace\f[] settings it is perfectly valid that +\f[C]ENOSPC\f[] / "out of space" / "no space left on device" be +returned. It is doing what was asked of it: filtering possible branches due to those settings. Only one error can be returned and if one of the reasons for filtering a -branch was \f[B]minfreespace\f[R] then it will be returned as such. -\f[B]moveonenospc\f[R] is only relevant to writing a file which is too +branch was \f[B]minfreespace\f[] then it will be returned as such. +\f[B]moveonenospc\f[] is only relevant to writing a file which is too large for the drive its currently on. .PP It is also possible that the filesystem selected has run out of inodes. -Use \f[C]df \-i\f[R] to list the total and available inodes per +Use \f[C]df\ \-i\f[] to list the total and available inodes per filesystem. .PP -If you don\[cq]t care about path preservation then simply change the -\f[C]create\f[R] policy to one which isn\[cq]t. -\f[C]mfs\f[R] is probably what most are looking for. -The reason it\[cq]s not default is because it was originally set to -\f[C]epmfs\f[R] and changing it now would change people\[cq]s setup. +If you don\[aq]t care about path preservation then simply change the +\f[C]create\f[] policy to one which isn\[aq]t. +\f[C]mfs\f[] is probably what most are looking for. +The reason it\[aq]s not default is because it was originally set to +\f[C]epmfs\f[] and changing it now would change people\[aq]s setup. Such a setting change will likely occur in mergerfs 3. .SS Why does the total available space in mergerfs not equal outside? .PP Are you using ext2/3/4? With reserve for root? mergerfs uses available space for statfs calculations. -If you\[cq]ve reserved space for root then it won\[cq]t show up. +If you\[aq]ve reserved space for root then it won\[aq]t show up. .PP -You can remove the reserve by running: \f[C]tune2fs \-m 0 \f[R] +You can remove the reserve by running: +\f[C]tune2fs\ \-m\ 0\ \f[] .SS Can mergerfs mounts be exported over NFS? .PP Yes, however if you do anything which may changes files out of band -(including for example using the \f[C]newest\f[R] policy) it will result -in \[lq]stale file handle\[rq] errors unless properly setup. +(including for example using the \f[C]newest\f[] policy) it will result +in "stale file handle" errors unless properly setup. .PP Be sure to use the following options: .IP \[bu] 2 @@ -2596,51 +2580,52 @@ to how Samba is setup in relation to permissions. Yes. .SS I notice massive slowdowns of writes when enabling cache.files. .PP -When file caching is enabled in any form (\f[C]cache.files!=off\f[R] or -\f[C]direct_io=false\f[R]) it will issue \f[C]getxattr\f[R] requests for -\f[C]security.capability\f[R] prior to \f[I]every single write\f[R]. +When file caching is enabled in any form (\f[C]cache.files!=off\f[] or +\f[C]direct_io=false\f[]) it will issue \f[C]getxattr\f[] requests for +\f[C]security.capability\f[] prior to \f[I]every single write\f[]. This will usually result in a performance degradation, especially when using a network filesystem (such as NFS or CIFS/SMB/Samba.) Unfortunately at this moment the kernel is not caching the response. .PP To work around this situation mergerfs offers a few solutions. .IP "1." 3 -Set \f[C]security_capability=false\f[R]. -It will short circuit any call and return \f[C]ENOATTR\f[R]. +Set \f[C]security_capability=false\f[]. +It will short circuit any call and return \f[C]ENOATTR\f[]. This still means though that mergerfs will receive the request before -every write but at least it doesn\[cq]t get passed through to the +every write but at least it doesn\[aq]t get passed through to the underlying filesystem. .IP "2." 3 -Set \f[C]xattr=noattr\f[R]. -Same as above but applies to \f[I]all\f[R] calls to getxattr. -Not just \f[C]security.capability\f[R]. -This will not be cached by the kernel either but mergerfs\[cq] runtime +Set \f[C]xattr=noattr\f[]. +Same as above but applies to \f[I]all\f[] calls to getxattr. +Not just \f[C]security.capability\f[]. +This will not be cached by the kernel either but mergerfs\[aq] runtime config system will still function. .IP "3." 3 -Set \f[C]xattr=nosys\f[R]. -Results in mergerfs returning \f[C]ENOSYS\f[R] which \f[I]will\f[R] be +Set \f[C]xattr=nosys\f[]. +Results in mergerfs returning \f[C]ENOSYS\f[] which \f[I]will\f[] be cached by the kernel. No future xattr calls will be forwarded to mergerfs. The downside is that also means the xattr based config and query -functionality won\[cq]t work either. +functionality won\[aq]t work either. .IP "4." 3 Disable file caching. -If you aren\[cq]t using applications which use \f[C]mmap\f[R] it\[cq]s +If you aren\[aq]t using applications which use \f[C]mmap\f[] it\[aq]s probably simpler to just disable it all together. -The kernel won\[cq]t send the requests when caching is disabled. +The kernel won\[aq]t send the requests when caching is disabled. .SS What are these .fuse_hidden files? .PP Please upgrade. mergerfs >= 2.26.0 will not have these temporary files. -See the notes on \f[C]unlink\f[R]. -.SS It\[cq]s mentioned that there are some security issues with mhddfs. What are they? How does mergerfs address them? +See the notes on \f[C]unlink\f[]. +.SS It\[aq]s mentioned that there are some security issues with mhddfs. +What are they? How does mergerfs address them? .PP mhddfs (https://github.com/trapexit/mhddfs) manages running as -\f[B]root\f[R] by calling +\f[B]root\f[] by calling getuid() (https://github.com/trapexit/mhddfs/blob/cae96e6251dd91e2bdc24800b4a18a74044f6672/src/main.c#L319) -and if it returns \f[B]0\f[R] then it will +and if it returns \f[B]0\f[] then it will chown (http://linux.die.net/man/1/chown) the file. -Not only is that a race condition but it doesn\[cq]t handle other +Not only is that a race condition but it doesn\[aq]t handle other situations. Rather than attempting to simulate POSIX ACL behavior the proper way to manage this is to use seteuid (http://linux.die.net/man/2/seteuid) and @@ -2651,10 +2636,10 @@ This is what mergerfs does and why mergerfs should always run as root. In Linux setreuid syscalls apply only to the thread. GLIBC hides this away by using realtime signals to inform all threads to change credentials. -Taking after \f[B]Samba\f[R], mergerfs uses -\f[B]syscall(SYS_setreuid,\&...)\f[R] to set the callers credentials for +Taking after \f[B]Samba\f[], mergerfs uses +\f[B]syscall(SYS_setreuid,...)\f[] to set the callers credentials for that thread only. -Jumping back to \f[B]root\f[R] as necessary should escalated privileges +Jumping back to \f[B]root\f[] as necessary should escalated privileges be needed (for instance: to clone paths between drives). .PP For non\-Linux systems mergerfs uses a read\-write lock and changes @@ -2664,10 +2649,10 @@ to change the processes credentials. So long as the other threads need to be user X they will take a readlock allowing multiple threads to share the credentials. Once a request comes in to run as user Y that thread will attempt a -write lock and change to Y\[cq]s credentials when it can. +write lock and change to Y\[aq]s credentials when it can. If the ability to give writers priority is supported then that flag will -be used so threads trying to change credentials don\[cq]t starve. -This isn\[cq]t the best solution but should work reasonably well +be used so threads trying to change credentials don\[aq]t starve. +This isn\[aq]t the best solution but should work reasonably well assuming there are few users. .SH SUPPORT .PP @@ -2675,7 +2660,7 @@ Filesystems are complex and difficult to debug. mergerfs, while being just a proxy of sorts, is also very difficult to debug given the large number of possible settings it can have itself and the massive number of environments it can run in. -When reporting on a suspected issue \f[B]please, please\f[R] include as +When reporting on a suspected issue \f[B]please, please\f[] include as much of the below information as possible otherwise it will be difficult or impossible to diagnose. Also please make sure to read all of the above documentation as it @@ -2685,45 +2670,41 @@ includes nearly every known system or user issue previously encountered. release (https://github.com/trapexit/mergerfs/releases) or have tried it in comparison. Old versions, which are often included in distros like Debian and Ubuntu, are not ever going to be updated and your bug may -have been addressed already.\f[R] +have been addressed already.\f[] .SS Information to include in bug reports .IP \[bu] 2 -Version of mergerfs: \f[C]mergerfs \-V\f[R] +Version of mergerfs: \f[C]mergerfs\ \-V\f[] .IP \[bu] 2 -mergerfs settings: from \f[C]/etc/fstab\f[R] or command line execution +mergerfs settings: from \f[C]/etc/fstab\f[] or command line execution .IP \[bu] 2 -Version of Linux: \f[C]uname \-a\f[R] +Version of Linux: \f[C]uname\ \-a\f[] .IP \[bu] 2 Versions of any additional software being used .IP \[bu] 2 List of drives, their filesystems, and sizes (before and after issue): -\f[C]df \-h\f[R] +\f[C]df\ \-h\f[] .IP \[bu] 2 -\f[B]All\f[R] information about the relevant branches and paths: +\f[B]All\f[] information about the relevant branches and paths: permissions, etc. .IP \[bu] 2 -A \f[C]strace\f[R] of the app having problems: -.RS 2 +A \f[C]strace\f[] of the app having problems: .IP \[bu] 2 -\f[C]strace \-f \-o /tmp/app.strace.txt \f[R] -.RE +\f[C]strace\ \-f\ \-o\ /tmp/app.strace.txt\ \f[] .IP \[bu] 2 -A \f[C]strace\f[R] of mergerfs while the program is trying to do -whatever it\[cq]s failing to do: -.RS 2 +A \f[C]strace\f[] of mergerfs while the program is trying to do whatever +it\[aq]s failing to do: .IP \[bu] 2 -\f[C]strace \-f \-p \-o /tmp/mergerfs.strace.txt\f[R] -.RE +\f[C]strace\ \-f\ \-p\ \ \-o\ /tmp/mergerfs.strace.txt\f[] .IP \[bu] 2 -\f[B]Precise\f[R] directions on replicating the issue. -Do not leave \f[B]anything\f[R] out. +\f[B]Precise\f[] directions on replicating the issue. +Do not leave \f[B]anything\f[] out. .IP \[bu] 2 Try to recreate the problem in the simplest way using standard programs. .SS Contact / Issue submission .IP \[bu] 2 github.com: https://github.com/trapexit/mergerfs/issues .IP \[bu] 2 -email: trapexit\[at]spawn.link +email: trapexit\@spawn.link .IP \[bu] 2 twitter: https://twitter.com/_trapexit .IP \[bu] 2 @@ -2753,24 +2734,36 @@ Ko\-Fi: https://ko\-fi.com/trapexit .IP \[bu] 2 Open Collective: https://opencollective.com/trapexit .IP \[bu] 2 -Bitcoin (BTC): 1DfoUd2m5WCxJAMvcFuvDpT4DR2gWX2PWb +Bitcoin (BTC): bc1qjwlywkqxgrxql3m7a7fvcsf3z3t98jvtekqp2j .IP \[bu] 2 -Bitcoin Cash (BCH): qrf257j0l09yxty4kur8dk2uma8p5vntdcpks72l8z +Bitcoin Cash (BCH): qrvymmkvuk7703m7cx0pqxc3mz4mmsn6ngn9xw52kc .IP \[bu] 2 -Ethereum (ETH): 0xb486C0270fF75872Fc51d85879b9c15C380E66CA +Bitcoin SV (BSV): 1FkFuxRtt3f8LbkpeUKRZq7gKJFzGSGgZV .IP \[bu] 2 -Litecoin (LTC): LW1rvHRPWtm2NUEMhJpP4DjHZY1FaJ1WYs +Bitcoin Gold (BTG): Gfk8QbMJFgpMTcY7uB63axy6HU7uTPPWNj .IP \[bu] 2 -Monero (XMR): -8AuU7PeK1fVhGP9yug8fdgKBssvUQoBVFKGhtT5DzWQt7fcTKC1SUx3Eb7xCAiVt3McWJp2Z9gX2wU7SPhh1GfWYBTCs6SS +Basic Attention Token (BAT): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +.IP \[bu] 2 +Chainlink (LINK): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +.IP \[bu] 2 +Dash (DASH): Xu2U3Nd3G4hM5TRQUBcP4DHJFzXH93xB84 .IP \[bu] 2 -Basic Attention Token (BAT): 0xE651d4900B4C305284Da43E2e182e9abE149A87A +Dogecoin (DOGE): DGFBPsRBYL8wHbgnvKbYkVn5FvAe854p1c +.IP \[bu] 2 +Ethereum (ETH): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +.IP \[bu] 2 +Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy .IP \[bu] 2 LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r .IP \[bu] 2 -Zcash (ZEC): t1ZwTgmbQF23DJrzqbAmw8kXWvU2xUkkhTt +Litecoin (LTC): LfL7jLNYuVpy7v5TyRyc3yRZ2uhqc4UoR3 +.IP \[bu] 2 +Monero (XMR): +45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f +.IP \[bu] 2 +Tezos (XTZ): tz1ZxerkbbALsuU9XGV9K9fFpuLWnKAGfc1C .IP \[bu] 2 -Zcoin (XZC): a8L5Vz35KdCQe7Y7urK2pcCGau7JsqZ5Gw +Zcash (ZEC): t1bjbVBK7tx9EGBrnD2wDfjGV9yZrcyfMmr .IP \[bu] 2 Other crypto currencies: contact me for address .SH LINKS From 6a9f7e6afce748941344116c99ea925543df453a Mon Sep 17 00:00:00 2001 From: Antonio SJ Musumeci Date: Mon, 8 Feb 2021 17:55:49 -0500 Subject: [PATCH 6/6] readme: update support section --- README.md | 36 +++++++++++++++++++++--------------- man/mergerfs.1 | 49 ++++++++++++++++++++++++++++++++----------------- 2 files changed, 53 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 5dd3aad3..673ecfcf 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ % mergerfs(1) mergerfs user manual % Antonio SJ Musumeci -% 2020-08-30 +% 2021-02-08 # NAME @@ -1283,28 +1283,34 @@ This software is free to use and released under a very liberal license (ISC). Th At the moment my preference would be GitHub Sponsors only because I am part of the matching program. That said please use whatever platform you prefer. -* PayPal: https://paypal.me/trapexit * GitHub Sponsors: https://github.com/sponsors/trapexit +* PayPal: https://paypal.me/trapexit * Patreon: https://www.patreon.com/trapexit -* SubscribeStar: https://www.subscribestar.com/trapexit * Ko-Fi: https://ko-fi.com/trapexit * Open Collective: https://opencollective.com/trapexit - -* Bitcoin (BTC): bc1qjwlywkqxgrxql3m7a7fvcsf3z3t98jvtekqp2j -* Bitcoin Cash (BCH): qrvymmkvuk7703m7cx0pqxc3mz4mmsn6ngn9xw52kc +* Bitcoin (BTC): bc1qu537hqlnmn2wawx9n7nws0dlkz55h0cd93ny28 +* Bitcoin Cash (BCH): bitcoincash:qqp0vh9v44us74gaggwjfv9y54zfjmmd7srlqxa3xt * Bitcoin SV (BSV): 1FkFuxRtt3f8LbkpeUKRZq7gKJFzGSGgZV -* Bitcoin Gold (BTG): Gfk8QbMJFgpMTcY7uB63axy6HU7uTPPWNj -* Basic Attention Token (BAT): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F -* Chainlink (LINK): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F -* Dash (DASH): Xu2U3Nd3G4hM5TRQUBcP4DHJFzXH93xB84 -* Dogecoin (DOGE): DGFBPsRBYL8wHbgnvKbYkVn5FvAe854p1c -* Ethereum (ETH): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +* Bitcoin Gold (BTG): AaPuJgJeohPjkB3LxJM6NKGnaHoRJ8ieT3 +* Litecoin (LTC): MJQzsHBdNnkyGqCFdcAdHYKugicBmfAXfQ +* Dogecoin (DOGE): DLJNLVe28vZ4SMQSxDJLBQBv57rGtUoWFh +* Ethereum (ETH): 0xB8d6d55c0319aacC327860d13f891427caEede7a + * Basic Attention Token (BAT): 0xB8d6d55c0319aacC327860d13f891427caEede7a + * Chainlink (LINK): 0xB8d6d55c0319aacC327860d13f891427caEede7a + * Reserve Rights (RSR): 0xB8d6d55c0319aacC327860d13f891427caEede7a + * Reef Finance (REEF): 0xB8d6d55c0319aacC327860d13f891427caEede7a + * Any ERC20 Token: 0xB8d6d55c0319aacC327860d13f891427caEede7a +* Ethereum Classic (ETC): 0x2B6054428e69a1201B6555f7a2aEc0Fba01EAD9F +* Dash (DASH): XvsFrohu8tbjA4E8p7xsc86E2ADxLHGXHL +* Monero (XMR): 45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f * Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy * LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r -* Litecoin (LTC): LfL7jLNYuVpy7v5TyRyc3yRZ2uhqc4UoR3 -* Monero (XMR): 45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f +* Ripple (XRP): r9f6aoxaGD8aymxqH89Ke1PCUPkNiFdZZC * Tezos (XTZ): tz1ZxerkbbALsuU9XGV9K9fFpuLWnKAGfc1C -* Zcash (ZEC): t1bjbVBK7tx9EGBrnD2wDfjGV9yZrcyfMmr +* Zcash (ZEC): t1Zo1GGn2T3GrhKvgdtnTsTnWu6tCPaCaHG +* DigiByte (DGB): Sb8r1qTrryY9Sp4YkTE1eeKEGVzgArnE5N +* Namecoin (NMC): NDzb9FkoptGu5QbgetCkodJqo2zE1cTwyb +* Vertcoin (VTC): 3PYdhokAGXJwWrwHRoTywxG4iUDk6EHjKe * Other crypto currencies: contact me for address diff --git a/man/mergerfs.1 b/man/mergerfs.1 index 12a3c2e8..993eb7e3 100644 --- a/man/mergerfs.1 +++ b/man/mergerfs.1 @@ -1,7 +1,7 @@ .\"t .\" Automatically generated by Pandoc 1.19.2.4 .\" -.TH "mergerfs" "1" "2020\-08\-30" "mergerfs user manual" "" +.TH "mergerfs" "1" "2021\-02\-08" "mergerfs user manual" "" .hy .SH NAME .PP @@ -2722,48 +2722,63 @@ At the moment my preference would be GitHub Sponsors only because I am part of the matching program. That said please use whatever platform you prefer. .IP \[bu] 2 -PayPal: https://paypal.me/trapexit -.IP \[bu] 2 GitHub Sponsors: https://github.com/sponsors/trapexit .IP \[bu] 2 -Patreon: https://www.patreon.com/trapexit +PayPal: https://paypal.me/trapexit .IP \[bu] 2 -SubscribeStar: https://www.subscribestar.com/trapexit +Patreon: https://www.patreon.com/trapexit .IP \[bu] 2 Ko\-Fi: https://ko\-fi.com/trapexit .IP \[bu] 2 Open Collective: https://opencollective.com/trapexit .IP \[bu] 2 -Bitcoin (BTC): bc1qjwlywkqxgrxql3m7a7fvcsf3z3t98jvtekqp2j +Bitcoin (BTC): bc1qu537hqlnmn2wawx9n7nws0dlkz55h0cd93ny28 .IP \[bu] 2 -Bitcoin Cash (BCH): qrvymmkvuk7703m7cx0pqxc3mz4mmsn6ngn9xw52kc +Bitcoin Cash (BCH): +bitcoincash:qqp0vh9v44us74gaggwjfv9y54zfjmmd7srlqxa3xt .IP \[bu] 2 Bitcoin SV (BSV): 1FkFuxRtt3f8LbkpeUKRZq7gKJFzGSGgZV .IP \[bu] 2 -Bitcoin Gold (BTG): Gfk8QbMJFgpMTcY7uB63axy6HU7uTPPWNj +Bitcoin Gold (BTG): AaPuJgJeohPjkB3LxJM6NKGnaHoRJ8ieT3 .IP \[bu] 2 -Basic Attention Token (BAT): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +Litecoin (LTC): MJQzsHBdNnkyGqCFdcAdHYKugicBmfAXfQ .IP \[bu] 2 -Chainlink (LINK): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +Dogecoin (DOGE): DLJNLVe28vZ4SMQSxDJLBQBv57rGtUoWFh .IP \[bu] 2 -Dash (DASH): Xu2U3Nd3G4hM5TRQUBcP4DHJFzXH93xB84 +Ethereum (ETH): 0xB8d6d55c0319aacC327860d13f891427caEede7a .IP \[bu] 2 -Dogecoin (DOGE): DGFBPsRBYL8wHbgnvKbYkVn5FvAe854p1c +Basic Attention Token (BAT): 0xB8d6d55c0319aacC327860d13f891427caEede7a .IP \[bu] 2 -Ethereum (ETH): 0x6241857fa5fb7667FB7a792b13E83fDEabe96f7F +Chainlink (LINK): 0xB8d6d55c0319aacC327860d13f891427caEede7a .IP \[bu] 2 -Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy +Reserve Rights (RSR): 0xB8d6d55c0319aacC327860d13f891427caEede7a .IP \[bu] 2 -LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r +Reef Finance (REEF): 0xB8d6d55c0319aacC327860d13f891427caEede7a +.IP \[bu] 2 +Any ERC20 Token: 0xB8d6d55c0319aacC327860d13f891427caEede7a +.IP \[bu] 2 +Ethereum Classic (ETC): 0x2B6054428e69a1201B6555f7a2aEc0Fba01EAD9F .IP \[bu] 2 -Litecoin (LTC): LfL7jLNYuVpy7v5TyRyc3yRZ2uhqc4UoR3 +Dash (DASH): XvsFrohu8tbjA4E8p7xsc86E2ADxLHGXHL .IP \[bu] 2 Monero (XMR): 45BBZMrJwPSaFwSoqLVNEggWR2BJJsXxz7bNz8FXnnFo3GyhVJFSCrCFSS7zYwDa9r1TmFmGMxQ2HTntuc11yZ9q1LeCE8f .IP \[bu] 2 +Filecoin (FIL): f1wpypkjcluufzo74yha7p67nbxepzizlroockgcy +.IP \[bu] 2 +LBRY Credits (LBC): bFusyoZPkSuzM2Pr8mcthgvkymaosJZt5r +.IP \[bu] 2 +Ripple (XRP): r9f6aoxaGD8aymxqH89Ke1PCUPkNiFdZZC +.IP \[bu] 2 Tezos (XTZ): tz1ZxerkbbALsuU9XGV9K9fFpuLWnKAGfc1C .IP \[bu] 2 -Zcash (ZEC): t1bjbVBK7tx9EGBrnD2wDfjGV9yZrcyfMmr +Zcash (ZEC): t1Zo1GGn2T3GrhKvgdtnTsTnWu6tCPaCaHG +.IP \[bu] 2 +DigiByte (DGB): Sb8r1qTrryY9Sp4YkTE1eeKEGVzgArnE5N +.IP \[bu] 2 +Namecoin (NMC): NDzb9FkoptGu5QbgetCkodJqo2zE1cTwyb +.IP \[bu] 2 +Vertcoin (VTC): 3PYdhokAGXJwWrwHRoTywxG4iUDk6EHjKe .IP \[bu] 2 Other crypto currencies: contact me for address .SH LINKS