diff --git a/mkdocs/docs/faq/configuration_and_policies.md b/mkdocs/docs/faq/configuration_and_policies.md index d50c65cb..c8ff5e01 100644 --- a/mkdocs/docs/faq/configuration_and_policies.md +++ b/mkdocs/docs/faq/configuration_and_policies.md @@ -79,28 +79,3 @@ If you wish to move files around or balance the pool you can: [mergerfs.balance](https://github.com/trapexit/mergerfs-tools/blob/master/src/mergerfs.balance). Keep in mind that this tool is really just an example of how to accomplish such a task. - - - -## Why are all my files ending up on 1 filesystem?! - -Did you start with empty filesystems? Did you explicitly configure a -`category.create` policy? Are you using an `existing path` / `path -preserving` policy? - -The default create policy is `epmfs`. That is a path preserving -algorithm. With such a policy for `mkdir` and `create` with a set of -empty filesystems it will select only 1 filesystem when the first -directory is created. Anything, files or directories, created in that -directory will be placed on the same branch because it is preserving -paths. - -This may catch new users off guard but this policy is the safest -policy to start with as it will not change the general layout of the -underlying filesystems. If you do not care about path preservation -(most shouldn't) and wish your files to be spread across all your -filesystems change to `rand`, `pfrd`, `mfs` or similar -[policy](../config/functions_categories_and_policies.md). If you do -want path preservation you'll need to perform the manual act of -creating paths on the filesystems you want the data to land on before -writing said data. diff --git a/mkdocs/docs/faq/technical_behavior_and_limitations.md b/mkdocs/docs/faq/technical_behavior_and_limitations.md index ec8d114a..26f40806 100644 --- a/mkdocs/docs/faq/technical_behavior_and_limitations.md +++ b/mkdocs/docs/faq/technical_behavior_and_limitations.md @@ -2,21 +2,26 @@ ## Do hardlinks work? -Yes. See also the option `inodecalc` for how inode values are -calculated. - -What mergerfs does not do is fake hard links across branches. Read -the section "rename & link" for how it works. - -Remember that hardlinks will NOT work across devices. That includes -between the original filesystem and a mergerfs pool, between two -separate pools of the same underlying filesystems, or bind mounts of -paths within the mergerfs pool. The latter is common when using Docker -or Podman. Multiple volumes (bind mounts) to the same underlying -filesystem are considered different devices. There is no way to link -between them. You should mount in the highest directory in the -mergerfs pool that includes all the paths you need if you want links -to work. +Yes. There is no option to enable or disable links. They are +fundimentally supported in compatible situations. That said the inode +of a file is not necessarily indicitive of two file names linking to +the same underlying data. See also the option `inodecalc` for how +inode values are calculated. + +What mergerfs does not do is fake hard links across branches. Read the +section [rename & link](../config/rename_and_link.md) for how it +works. + +Remember that hardlinks will **NOT** work across devices. That +includes between the original filesystem and a mergerfs pool, between +two separate pools of the same underlying filesystems, or bind mounts +of paths within the mergerfs pool. The latter is common when using +Docker or Podman. Multiple volumes (bind mounts) to the same +underlying filesystem are considered different devices. There is no +way to link or rename between them. You should mount in the highest +directory in the mergerfs pool that includes all the paths you need if +you want links and rename to work. + ## How does mergerfs handle moving and copying of files? @@ -27,8 +32,8 @@ individual filesystem functions. A "move" can include a "copy" so lets describe copy first. -When an application copies a file from source to destination it can do -so in a number of ways but the basics are the following. +When an application copies a file from "source" to "destination" it +can do so in a number of ways but the basics are the following. 1. `open` the source file. 2. `create` the destination file. @@ -58,13 +63,16 @@ some low level settings for the operating system. All of this means that mergerfs can not make decisions when a file is created based on file size or the source of the data. That information is simply not available. At best mergerfs could respond to files -reaching a certain size when writing data or when a file is closed. +reaching a certain size when writing data, a file is closed, or +renamed. Related: if a user wished to have mergerfs perform certain activities based on the name of a file it is common and even best practice for a program to write to a temporary file first and then rename to its final destination. That temporary file name will typically be random -and have no indication of the type of file being written. +and have no indication of the type of file being written. At best +something could be done on rename. + ## Does FICLONE or FICLONERANGE work? @@ -75,26 +83,27 @@ an error back to the application making the request. Should FUSE gain the ability mergerfs will be updated to support it. + ## Why do I get an "out of space" / "no space left on device" / ENOSPC error even though there appears to be lots of space available? First make sure you've read the sections above about policies, path -preservation, branch filtering, and the options **minfreespace**, -**moveonenospc**, **statfs**, and **statfs_ignore**. +preservation, branch filtering, and the options `minfreespace`, +`moveonenospc`, `statfs`, and `statfs_ignore`. mergerfs is simply presenting a union of the content within multiple branches. The reported free space is an aggregate of space available -within the pool (behavior modified by **statfs** and -**statfs_ignore**). It does not represent a contiguous space. In the +within the pool (behavior modified by `statfs` and +`statfs_ignore`). It does not represent a contiguous space. In the same way that read-only filesystems, those with quotas, or reserved space report the full theoretical space available. Due to path preservation, branch tagging, read-only status, and -**minfreespace** settings it is perfectly valid that `ENOSPC` / "out -of space" / "no space left on device" be returned. It is doing what -was asked of it: filtering possible branches due to those -settings. Only one error can be returned and if one of the reasons for -filtering a branch was **minfreespace** then it will be returned as -such. **moveonenospc** is only relevant to writing a file which is too +`minfreespace` settings it is perfectly valid that `ENOSPC` / "out of +space" / "no space left on device" be returned. It is doing what was +asked of it: filtering possible branches due to those settings. Only +one error can be returned and if one of the reasons for filtering a +branch was `minfreespace` then it will be returned as +such. `moveonenospc` is only relevant to writing a file which is too large for the filesystem it's currently on. It is also possible that the filesystem selected has run out of @@ -107,6 +116,7 @@ looking for. The reason it's not default is because it was originally set to `epmfs` and changing it now would change people's setup. Such a setting change will likely occur in mergerfs 3. + ## Why does the total available space in mergerfs not equal outside? Are you using ext2/3/4? With reserve for root? mergerfs uses available @@ -115,6 +125,7 @@ it won't show up. You can remove the reserve by running: `tune2fs -m 0 ` + ## I notice massive slowdowns of writes when enabling cache.files. When file caching is enabled in any form (`cache.files!=off`) it will @@ -141,32 +152,6 @@ To work around this situation mergerfs offers a few solutions. `mmap` it's probably simpler to just disable it altogether. The kernel won't send the requests when caching is disabled. -## Why can't I see my files / directories? - -It's almost always a permissions issue. Unlike mhddfs and -unionfs-fuse, which runs as root and attempts to access content as -such, mergerfs always changes its credentials to that of the -caller. This means that if the user does not have access to a file or -directory than neither will mergerfs. However, because mergerfs is -creating a union of paths it may be able to read some files and -directories on one filesystem but not another resulting in an -incomplete set. - -Whenever you run into a split permission issue (seeing some but not -all files) try using -[mergerfs.fsck](https://github.com/trapexit/mergerfs-tools) tool to -check for and fix the mismatch. If you aren't seeing anything at all -be sure that the basic permissions are correct. The user and group -values are correct and that directories have their executable bit -set. A common mistake by users new to Linux is to `chmod -R 644` when -they should have `chmod -R u=rwX,go=rX`. - -If using a network filesystem such as NFS or SMB (Samba) be sure to -pay close attention to anything regarding permissioning and -users. Root squashing and user translation for instance has bitten a -few mergerfs users. Some of these also affect the use of mergerfs from -container platforms such as Docker. - ## Why use FUSE? Why not a kernel based solution? diff --git a/mkdocs/docs/faq/why_isnt_it_working.md b/mkdocs/docs/faq/why_isnt_it_working.md new file mode 100644 index 00000000..e6075ffc --- /dev/null +++ b/mkdocs/docs/faq/why_isnt_it_working.md @@ -0,0 +1,112 @@ +# "Why isn't it working?" + +## I modified mergerfs' config but it still behaves the same. + +mergerfs, like most filesystems, are given their options/arguments +at mount time. Unlike most filesystems, mergerfs also has the ability +to modify certain options at [runtime](../runtime_interfaces.md). + +That said: mergerfs does not actively try to monitor typical methods +of configuration (nor should it.) As such if changes are made to +`/etc/fstab`, a systemd unit file, etc. it will have no knowledge of +those changes. It is the user's responsibility to +[restart](../setup/upgrade.md) mergerfs to pick up the changes or use +the [runtime interface](../runtime_interfaces.md). + + +## Why are all my files ending up on 1 filesystem?! + +Did you start with empty filesystems? Did you explicitly configure a +`category.create` policy? Are you using an `existing path` / `path +preserving` policy? + +The default create policy is `epmfs`. That is a path preserving +algorithm. With such a policy for `mkdir` and `create` with a set of +empty filesystems it will select only 1 filesystem when the first +directory is created. Anything, files or directories, created in that +directory will be placed on the same branch because it is preserving +paths. That is the expected behavior. + +This may catch new users off guard but this policy is the safest +policy to start with as it will not change the general layout of the +underlying branches. If you do not care about path preservation ([most +should +not](configuration_and_policies.md#how-can-i-ensure-files-are-collocated-on-the-same-branch)) +and wish your files to be spread across all your filesystems change to +`pfrd`, `rand`, `mfs` or similar +[policy](../config/functions_categories_and_policies.md). + + +## Why isn't the create policy working? + +It probably is. The policies rather straight forward and well tested. + +First, confirm the policy is configured as expected by using the +[runtime interface](../runtime_interfaces.md). + +```shell +$ getfattr -n user.mergerfs.category.create /mnt/mergerfs/.mergerfs +# file: mnt/mergerfs/.mergerfs +user.mergerfs.category.create="mfs" +``` + +Second, as discussed in the [support](../support.md) section, test the +behavior using simple command line tools such as `touch` and then see +where it was created. + + +```shell +$ touch /mnt/mergerfs/new-file +$ getfattr -n user.mergerfs.allpaths /mnt/mergerfs/new-file +# file: mnt/mergerfs/new-file +user.mergerfs.allpaths="/mnt/hdd/drive1/new-file" +``` + +If the location of the file is where it should be according to the +state of the system at the time and the policy selected then the +"problem" lies elsewhere. + +[Keep in +mind](technical_behavior_and_limitations.md/#how-does-mergerfs-handle-moving-and-copying-of-files) +that files, when created, have no size. If a number of files are +created at the same time, for example by a program downloading +numerous files like a BitTorrent client, then depending on the policy +the files could be created on the same branch. As the files are +written to, or resized immediately afterwards to the total size of the +file being downloaded, the files will take up more space but there is +no mechanism to move them as they grow. Nor would it be a good idea to +do so as it would be expensive to continuously calculate their size and +perform the move while the file is still being written to. As such an +imbalance will occur that wouldn't if the files had been downloaded +one at a time. + +If you wish to reduce the likelihood of this happening a policy that +does not make decisions on available space alone such as `pfrd` or +`rand` should be used. + + +## Why can't I see my files / directories? + +It's almost always a permissions issue. Unlike mhddfs and +unionfs-fuse, which accesses content as root, mergerfs always changes +its credentials to that of the caller. This is done as it is the only +properly secure way to manage permissions. This means that if the user +does not have access to a file or directory than neither will +mergerfs. However, because mergerfs is creating a union of paths it +may be able to read some files and directories on one filesystem but +not another resulting in an incomplete set. And if one of the branches +it can access is empty then it will return an empty list. + +Try using [mergerfs.fsck](https://github.com/trapexit/mergerfs-tools) +tool to check for and fix inconsistencies in permissions. If you +aren't seeing anything at all be sure that the basic permissions are +correct. The user and group values are correct and that directories +have their executable bit set. A common mistake by users new to Linux +is to `chmod -R 644` when they should have `chmod -R u=rwX,go=rX`. + +If using a [network filesystem](../remote_filesystems.md) such as NFS +or SMB (Samba) be sure to pay close attention to anything regarding +permissioning and users. Root squashing and user translation for +instance has bitten a few mergerfs users. Some of these also affect +the use of mergerfs from [container platforms such as +Docker.](compatibility_and_integration.md) diff --git a/mkdocs/docs/setup/upgrade.md b/mkdocs/docs/setup/upgrade.md index 43dcfbf6..596638a9 100644 --- a/mkdocs/docs/setup/upgrade.md +++ b/mkdocs/docs/setup/upgrade.md @@ -4,11 +4,12 @@ mergerfs can be upgraded live by mounting on top of the previous instance. Simply install the new version of mergerfs and follow the instructions below. -Run mergerfs again or if using `/etc/fstab` call for it to mount -again. Existing open files and such will continue to work fine though -they won't see runtime changes since any such change would be the new -mount. If you plan on changing settings with the new mount you should -/ could apply those before mounting the new version. +Run mergerfs again or if using `/etc/fstab` (or systemd mount file) +call for it to mount again. Existing open files and directories will +continue to work fine though they won't see any differences that the +new version would provide since it is still using the previous +instance. If you plan on changing settings with the new mount you +should / could apply those before mounting the new version. ``` $ sudo mount /mnt/mergerfs @@ -23,3 +24,6 @@ restarted. To work around this you can use a "lazy umount". Before mounting over top the mount point with the new instance of mergerfs issue: `umount -l `. Or you can let mergerfs do it by setting the option `lazy-umount-mountpoint=true`. + +If the intent is to change settings at runtime then the [runtime +interface](../runtime_interfaces.md) should be used. diff --git a/mkdocs/mkdocs.yml b/mkdocs/mkdocs.yml index 48fccba4..18aac855 100644 --- a/mkdocs/mkdocs.yml +++ b/mkdocs/mkdocs.yml @@ -97,6 +97,7 @@ nav: - faq/compatibility_and_integration.md - faq/recommendations_and_warnings.md - faq/technical_behavior_and_limitations.md + - faq/why_isnt_it_working.md - faq/limit_drive_spinup.md - related_projects.md - media_and_publicity.md