diff --git a/mkdocs/docs/config/rename_and_link.md b/mkdocs/docs/config/rename_and_link.md index c2e26d79..a8b234cc 100644 --- a/mkdocs/docs/config/rename_and_link.md +++ b/mkdocs/docs/config/rename_and_link.md @@ -1,12 +1,5 @@ # rename and link -**NOTE:** If you're receiving errors from software when files are -moved / renamed / linked then you should consider changing the create -policy to one which is **not** path preserving, enabling -`ignorepponrename`, or contacting the author of the offending software -and requesting that `EXDEV` (cross device / improper link) be properly -handled. - `rename` and `link` are arguably the most complicated functions to create in a union filesystem. `rename` only works within a single filesystem or device. If a `rename` can not be done due to the source @@ -42,9 +35,9 @@ below. * Using the **rename** policy get the list of files to rename * Using the **getattr** policy get the target path * For each file attempt rename: - * If the source branch != target branch: - * Clone target path from target branch to source branch - * rename + * If the target path does not exist on the source branch: + * Clone target path from target branch to source branch + * rename * If **any** of the renames succeed the higher level rename is considered a success * If **no** renames succeed the first error encountered will be returned * On success: @@ -58,3 +51,10 @@ The above behavior will help minimize the likelihood of EXDEV being returned but it will still be possible. **link** uses the same strategy but without the removals. + + +## Additional Reading + +* [Do hardlinks work?](../faq/technical_behavior_and_limitations.md#do-hardlinks-work) +* [How does mergerfs handle moving and copying of files?](../faq/technical_behavior_and_limitations.md#how-does-mergerfs-handle-moving-and-copying-of-files) +* [Why are file moves and renames failing?](../faq/why_isnt_it_working.md#why-are-file-moves-renames-linkshardlinks-failing) diff --git a/mkdocs/docs/faq/configuration_and_policies.md b/mkdocs/docs/faq/configuration_and_policies.md index c8ff5e01..ec8e6481 100644 --- a/mkdocs/docs/faq/configuration_and_policies.md +++ b/mkdocs/docs/faq/configuration_and_policies.md @@ -78,4 +78,7 @@ If you wish to move files around or balance the pool you can: 3. Use [mergerfs.balance](https://github.com/trapexit/mergerfs-tools/blob/master/src/mergerfs.balance). Keep in mind that this tool is really just an example of how to - accomplish such a task. + accomplish such a task. The tool does not keep track of links so + you may encounter an increase in used space if you rely on links to + reduce redundency. However, you can run a file dedup program like + [rdfind](https://github.com/pauldreik/rdfind) to restore the links. diff --git a/mkdocs/docs/faq/technical_behavior_and_limitations.md b/mkdocs/docs/faq/technical_behavior_and_limitations.md index 26f40806..39f32146 100644 --- a/mkdocs/docs/faq/technical_behavior_and_limitations.md +++ b/mkdocs/docs/faq/technical_behavior_and_limitations.md @@ -2,11 +2,11 @@ ## Do hardlinks work? -Yes. There is no option to enable or disable links. They are -fundimentally supported in compatible situations. That said the inode -of a file is not necessarily indicitive of two file names linking to -the same underlying data. See also the option `inodecalc` for how -inode values are calculated. +Yes. There is no option to enable or disable links (or renames.) They +are fundamentally supported in compatible situations. That said the +inode of a file is not necessarily indicative of two file names +linking to the same underlying data. See also the option `inodecalc` +for how inode values are calculated. What mergerfs does not do is fake hard links across branches. Read the section [rename & link](../config/rename_and_link.md) for how it @@ -206,7 +206,7 @@ removed to simplify the codebase. mergerfs is a multithreaded application in order to handle requests from the kernel concurrently. Each FUSE message has a header with certain details about the request include the process ID (pid) of the -requestion application, the process' effective user id (uid), and +requesting application, the process' effective user id (uid), and group id (gid). To ensure proper POSIX filesystem behavior and security mergerfs must change its identity to match that of the requester when performing the core filesystem function on the diff --git a/mkdocs/docs/faq/why_isnt_it_working.md b/mkdocs/docs/faq/why_isnt_it_working.md index e6075ffc..8b9ec6e4 100644 --- a/mkdocs/docs/faq/why_isnt_it_working.md +++ b/mkdocs/docs/faq/why_isnt_it_working.md @@ -28,13 +28,13 @@ directory will be placed on the same branch because it is preserving paths. That is the expected behavior. This may catch new users off guard but this policy is the safest -policy to start with as it will not change the general layout of the -underlying branches. If you do not care about path preservation ([most -should +policy to have as default as it will not change the general layout of +the underlying branches. If you do not care about path preservation +([most should not](configuration_and_policies.md#how-can-i-ensure-files-are-collocated-on-the-same-branch)) -and wish your files to be spread across all your filesystems change to -`pfrd`, `rand`, `mfs` or similar -[policy](../config/functions_categories_and_policies.md). +and wish your files to be spread across all your filesystems change +`create.category` to `pfrd`, `rand`, `mfs` or a similarly non-path +restrictive [policy](../config/functions_categories_and_policies.md). ## Why isn't the create policy working? @@ -75,10 +75,10 @@ the files could be created on the same branch. As the files are written to, or resized immediately afterwards to the total size of the file being downloaded, the files will take up more space but there is no mechanism to move them as they grow. Nor would it be a good idea to -do so as it would be expensive to continuously calculate their size and -perform the move while the file is still being written to. As such an -imbalance will occur that wouldn't if the files had been downloaded -one at a time. +do so as it would be expensive to continuously calculate their size +and perform the move while the file is still being written to. As such +an imbalance will occur that wouldn't if the files had been created +and resized one at a time. If you wish to reduce the likelihood of this happening a policy that does not make decisions on available space alone such as `pfrd` or @@ -110,3 +110,108 @@ permissioning and users. Root squashing and user translation for instance has bitten a few mergerfs users. Some of these also affect the use of mergerfs from [container platforms such as Docker.](compatibility_and_integration.md) + + +## Why are file moves, renames, links/hardlinks failing? + +mergerfs fully supports rename and link functions to the degree that +is possible in a union filesystem. Problems related to rename and link +are almost exclusively due to incorrect setup of software or poorly +written software. + + +### Docker, Podman, and other container runtimes + +Keep in mind that [rename and link](../config/rename_and_link.md) will +**NOT** work across devices. That includes between the original +filesystem and a mergerfs pool, between two separate pools of the same +underlying filesystems, or bind mounts of paths within the mergerfs +pool. The latter is common when using Docker, Podman, or other +container runtimes. Multiple volumes (bind mounts) to the same +underlying filesystem are considered different devices. There is no +way to link or rename between them despite them binding to the same +underlying filesystem. Neither mergerfs or any other filesystem has a +say in this behavior as the kernel is erroring out before the request +even gets to the filesystems in question. + +To work around this situation the highest directory in the mergerfs +pool that includes all the paths you need is what should be bind'ed +into the container if you want links and rename to work between the +paths in question. + +This is a common problem for users of Radarr, Sonarr, Lidarr, +etc. when they use the default volume mount options suggested by +projects like +[LinuxServer.io](https://docs.linuxserver.io/images/docker-radarr/?h=radarr#docker-compose-recommended-click-here-for-more-info). Do +not use separate "movies" and "downloads" volumes. + +``` +# NO +volumes: + - /mnt/pool/movies:/movies + - /mnt/pool/downloads:/downloads + +# YES +volumes: + - /mnt/pool:/media +# Then use /media/movies and /media/downloads in your software. +``` + + +### Path-Preserving Create Policies + +As described in the [rename and link](../config/rename_and_link.md) +docs if a path-preserving create +[policy](../config/functions_categories_and_policies.md) is configured +then mergerfs, in order to follow that restrictive policy, must return +an error +([EXDEV](https://man7.org/linux/man-pages/man2/rename.2.html)) if the +target path of a rename or link is not found on the target branch. + +``` +EXDEV oldpath and newpath are not on the same mounted filesystem. + (Linux permits a filesystem to be mounted at multiple + points, but rename() does not work across different mount + points, even if the same filesystem is mounted on both.) +``` + +[Most +users](configuration_and_policies.md#how-can-i-ensure-files-are-collocated-on-the-same-branch) +do not have a need for path-preserving policies but if you intend on +using path-preserving policies you must accept that `EXDEV` errors are +more likely to occur. You could enable +[ignorepponrename](../config/options.md) however doing so would +contradict the intention of using a path restrictive policy. + + +### Poorly written software + +There are some pieces of software that do not properly handle `EXDEV` +errors which `rename` and `link` may return even though they are +perfectly valid situations and do not indicate actual device, +filesystem, or OS problems. The error will rarely be returned by +mergerfs if using a non-path preserving +[policy](../config/functions_categories_and_policies.md) however there +are edge cases where it may (such as mounting another filesystem into +the mergerfs pool.) If you do not care about path preservation set the +mergerfs policy to the non-path preserving variant. For example: `-o +category.create=pfrd`. Ideally the offending software would be fixed and +it is recommended that if you run into this problem you contact the +software's author and request proper handling of `EXDEV` errors. + + +### Additional Reading + +* [rename and link](../config/rename_and_link.md) +* [Do hardlinks work?](technical_behavior_and_limitations.md#do-hardlinks-work) +* [How does mergerfs handle moving and copying of files?](technical_behavior_and_limitations.md#how-does-mergerfs-handle-moving-and-copying-of-files) +* [Moving files and directories fails with Samba](../known_issues_bugs.md#moving-files-and-directories-fails-with-samba) + + +## Why is rtorrent failing with ENODEV (No such device)? + +Be sure to set +[cache.files=partial|full|auto-full|per-process](../config/cache.md) +or use Linux kernel v6.6 or above. rtorrent and some other +applications use [mmap](http://linux.die.net/man/2/mmap) to read and +write to files and offer no fallback to traditional methods. diff --git a/mkdocs/docs/known_issues_bugs.md b/mkdocs/docs/known_issues_bugs.md index 2465a33f..7575f00d 100644 --- a/mkdocs/docs/known_issues_bugs.md +++ b/mkdocs/docs/known_issues_bugs.md @@ -71,6 +71,8 @@ Use `cache.files=off` and/or `dropcacheonclose=true`. See the section on [page caching](config/cache.md). +## 3rd Party Software + ### NFS clients returning ESTALE / Stale file handle NFS generally does not like out of band changes. Take a look at the @@ -78,55 +80,34 @@ section on NFS in the [remote-filesystems](remote_filesystems.md) for more details. -### rtorrent fails with ENODEV (No such device) - -Be sure to set -[cache.files=partial|full|auto-full|per-process](config/cache.md) -or use Linux kernel v6.6 or above. rtorrent and some other -applications use [mmap](http://linux.die.net/man/2/mmap) to read and -write to files and offer no fallback to traditional methods. - - -### Plex / Jellyfin doesn't work with mergerfs +### SQLite3, Plex, Jellyfin do not work with mergerfs It does. If you're trying to put the software's config / metadata / database on mergerfs you can't set [cache.files=off](config/cache.md) (unless you use Linux v6.6 or -above) because Plex is using **sqlite3** with **mmap** enabled. +above) because they are using **sqlite3** with **mmap** enabled. That said it is recommended that config and runtime files be stored on -SSDs on a regular filesystem for performance reasons and if you are -using HDDs in your pool to help limit spinup. +SSDs on a regular filesystem for performance reasons. See [What should +mergerfs NOT be used for?](faq/recommendations_and_warnings.md). Other software that leverages **sqlite3** which require **mmap** -includes Radarr, Sonarr, Lidarr. +includes Radarr, Sonarr, and Lidarr. That said many programs use +**sqlite3** and do not require **mmap**. It is recommended that you reach out to the developers of the software -you're having troubles with and asking them to add a fallback to +you are having troubles with and asking them to add a fallback to regular file IO when **mmap** is unavailable. It is not only more compatible and resilient but also can be more performant in certain situations. If the issue is that quick scanning doesn't seem to pick up media then -be sure to set `func.getattr=newest`, though generally, a full scan -will pick up all media anyway. - - -### When a program tries to move or rename a file it fails - -Please read the docs regarding [rename and -link](config/rename_and_link.md). - -The problem is that many applications do not properly handle `EXDEV` -errors which `rename` and `link` may return even though they are -perfectly valid situations which do not indicate actual device, -filesystem, or OS errors. The error will only be returned by mergerfs -if using a path preserving policy as described in the policy section -above. If you do not care about path preservation simply change the -mergerfs policy to the non-path preserving version. For example: `-o -category.create=mfs` Ideally the offending software would be fixed and -it is recommended that if you run into this problem you contact the -software's author and request proper handling of `EXDEV` errors. +be sure to set `func.getattr=newest`. That said a full scan will pick +up all media and it will put less load on the host to use time based +library scans or to configure downloading software to trigger a scan +when files are added to the pool. See [Does inotify and fanotify +work?](faq/compatibility_and_integration.md#does-inotify-and-fanotify-work) +for more details. ### my 32bit software has problems diff --git a/mkdocs/docs/quickstart.md b/mkdocs/docs/quickstart.md index e0dcbcca..f5a210c0 100644 --- a/mkdocs/docs/quickstart.md +++ b/mkdocs/docs/quickstart.md @@ -7,31 +7,35 @@ First ensure you have the [latest version installed](setup/installation.md). ## Configuration -mergerfs has many options and effectively all of them are functional -in nature. What that means is that there is no "best" or "fastest" -configuration. No "make faster" options. Everything changes -behavior. Sometimes those changes in behavior affect performance. +mergerfs has many [options](config/options.md) and effectively all of +them are functional in nature. What that means is that there is no +"best" or "fastest" configuration. No "make faster" +options. Everything changes behavior. Sometimes those changes in +behavior affect performance. + +If you don't already know that you have special requirements then use +one of the following option sets as it will cover the common use +cases. -That said: If you don't already know that you have a special use case -then use one of the following option sets as it will cover most casual -usecases. ### You use Linux v6.6 or above * cache.files=off -* category.create=mfs +* category.create=pfrd * dropcacheonclose=false -In previous versions of Linux it was unable to support `mmap` if page -caching was disabled (ie: `cache.files=off`). However, it now will -enable page caching if needed for a particular file if mmap is -requested. +In previous versions of Linux it was unable to support `mmap` if [page +caching](config/cache.md) was disabled (ie: +`cache.files=off`). However, it now will enable page caching if needed +for a particular file if `mmap` is requested. `mmap` is needed by certain software to read and write to a file. However, many software could work without it and fail to have proper error handling. Many programs that use sqlite3 will require -`mmap` despite sqlite3 working perfectly fine without it (and in some -cases can be more performant with regular file IO.) +`mmap` despite [sqlite3 working perfectly +fine](known_issues_bugs.md#sqlite3-plex-jellyfin-do-not-work-with-mergerfs) +without it (and in some cases can be more performant with regular file +IO.) ### You use Linux v6.5 or below @@ -39,14 +43,14 @@ cases can be more performant with regular file IO.) #### You need `mmap` (used by rtorrent and many sqlite3 base software) * cache.files=auto-full -* category.create=mfs +* category.create=pfrd * dropcacheonclose=true #### You don't need `mmap` * cache.files=off -* category.create=mfs +* category.create=pfrd * dropcacheonclose=false @@ -55,14 +59,14 @@ cases can be more performant with regular file IO.) ### Command Line ``` -mergerfs -o cache.files=off,dropcacheonclose=false,category.create=mfs /mnt/hdd0:/mnt/hdd1 /media +mergerfs -o cache.files=off,dropcacheonclose=false,category.create=pfrd /mnt/hdd0:/mnt/hdd1 /media ``` ### /etc/fstab ``` -/mnt/hdd0:/mnt/hdd1 /media mergerfs cache.files=off,dropcacheonclose=false,category.create=mfs 0 0 +/mnt/hdd0:/mnt/hdd1 /media mergerfs cache.files=off,dropcacheonclose=false,category.create=pfrd 0 0 ``` ### /etc/fstab w/ config file @@ -81,7 +85,7 @@ For more complex setups it can be useful to separate out the config. ```ini title="media.ini" linenums="1" cache.files=off -category.create=mfs +category.create=pfrd dropcacheonclose=false ``` @@ -114,7 +118,7 @@ KillMode=none ExecStart=/usr/bin/mergerfs \ -f \ -o cache.files=off \ - -o category.create=mfs \ + -o category.create=pfrd \ -o dropcacheonclose=false \ /mnt/hdd0:/mnt/hdd1 \ /media @@ -177,7 +181,7 @@ KillMode=none ExecStart=/usr/bin/mergerfs \ -f \ -o cache.files=off \ - -o category.create=mfs \ + -o category.create=pfrd \ -o dropcacheonclose=false \ /mnt/hdd0:/mnt/hdd1 \ /media diff --git a/mkdocs/docs/remote_filesystems.md b/mkdocs/docs/remote_filesystems.md index 377959a0..3ac128bf 100644 --- a/mkdocs/docs/remote_filesystems.md +++ b/mkdocs/docs/remote_filesystems.md @@ -1,19 +1,33 @@ # Remote Filesystems -Many users ask about compatibility with remote filesystems. This -section is to describe any known issues or quirks when using mergerfs -with common remote filesystems. +This section is to describe any known issues or quirks when using +mergerfs with common remote filesystems. -Keep in mind that, like with caching, it is **NOT** a good idea to -change the contents of the remote filesystem +There are two ways to use mergerfs with a network filesystem. One is +to use it within the pool as a branch and one is the exporting of a +mergerfs mount. + + +## General Notes + +### Do not change things out-of-band + +Keep in mind that, like when using +[caching](faq/usage_and_functionality.md#can-filesystems-still-be-used-directly-outside-of-mergerfs-while-pooled), +it is **NOT** a good idea to change the contents of the remote +filesystem [out-of-band](https://en.wikipedia.org/wiki/Out-of-band). Meaning that you should not change the contents of the underlying filesystems or -mergerfs on the server hosting the remote filesystem. Doing so can -lead to weird behavior, inconsistency, errors, and even data -corruption should multiple programs try to write or read the same data -at the same time. This isn't to say you can't do it or that data -corruption is likely but it _could_ happen. It is better to always use -the remote filesystem. Even on the machine serving it. +mergerfs on the server hosting the remote filesystem. Files should +exclusively be interacted with through the network filesystem. Doing +otherwise can lead to weird behavior, inconsistency, errors, and even +data corruption should multiple programs try to write or read the same +data at the same time. This isn't to say you can't do it or that data +corruption is likely but it _could_ happen. If you're only reading +from those filesystems there is little risk but writing or modifying +file layout will be a problem. Particularly with NFS. It is better to +always use the remote filesystem. Even on the machine serving it. + ## NFS @@ -22,44 +36,57 @@ remote filesystem on Unix/POSIX systems. Due to how NFS works there are some settings which need to be set in order for mergerfs to work with it. -It should be noted that NFS and FUSE (the technology mergerfs uses) do -not work perfectly with one another due to certain design choices in -FUSE (and mergerfs.) Due to these issues, it is generally recommended -to use SMB when possible. That said mergerfs should generally work as -an export of NFS and issues discovered should still be reported. - -To ensure compatibility between mergerfs and NFS use the following -settings. +### NFS as branch mergerfs settings: -* `noforget` -* `inodecalc=path-hash` +* No special settings need to be set NFS export settings: -* `fsid=UUID` -* `no_root_squash` +* `no_root_squash`: mergerfs must be able to make requests that only + root is permissioned to make. If any root squashing occurs, just as + running mergerfs as a non-root user, there is no guarantee that + mergerfs will be able to do what it needs to manage the underlying + filesystem. + + +### NFS exporting mergerfs + +It should be noted that NFS and FUSE (the technology mergerfs uses) do +not work perfectly with one another. They largely can be worked around +but if you run into problems it may be worth trying Samba/SMB. + +**mergerfs settings:** + +* `noforget` +* `inodecalc=path-hash` `noforget` is needed because NFS uses the `name_to_handle_at` and `open_by_handle_at` functions which allow a program to keep a reference to a file without technically having it open in the typical sense. The problem is that FUSE has no way to know that NFS has a handle that it will later use to open the file again. As a result, it -is possible for the kernel to tell mergerfs to forget about the node -and should NFS ever ask for that node's details in the future it would -have nothing to respond with. Keeping nodes around forever is not -ideal but at the moment the only way to manage the situation. +is possible for the kernel to tell mergerfs to forget about the file +node and should NFS ever ask for that node's details in the future it +would have nothing to respond with. Keeping nodes around forever is +not ideal but at the moment the only way to manage the situation. `inodecalc=path-hash` is needed because NFS is sensitive to out-of-band changes. FUSE doesn't care if a file's inode value changes but NFS, being stateful, does. So if you used the default inode -calculation algorithm then it is possible that if you changed a file -or updated a directory the file mergerfs will use will be on a -different branch and therefore the inode would change. This isn't an -ideal solution and others are being considered but it works for most +calculation algorithm it is possible that if you changed a file or +updated a directory the file mergerfs will use will be on a different +branch and therefore the inode would change. This isn't an ideal +solution and others are being considered but it works for most situations. +**NFS export settings:** + +* `fsid=UUID` +* `no_root_squash` + + `fsid=UUID` is needed because FUSE filesystems don't have different `st_dev` values which can cause issues when exporting. The easiest thing to do is set each mergerfs export `fsid` to some random @@ -67,10 +94,11 @@ value. An easy way to generate a random value is to use the command line tool `uuid` or `uuidgen` or through a website such as [uuidgenerator.net](https://www.uuidgenerator.net/). -`no_root_squash` is not strictly necessary but can lead to confusing -permission and ownership issues if root squashing is enabled. mergerfs -needs to run certain commands as `root` and if root squash is enabled -it NFS will tell mergerfs a non-root user is making certain calls. +`no_root_squash` is required for the same reason mergerfs needs to run +as `root`. Certain behaviors of mergerfs require control over the +filesystem that only `root` can preform. If squashing is enabled, or +mergerfs was running as non-root, it would be unable to perform +certain function and you will receive permission errors. ## SMB / CIFS @@ -81,11 +109,20 @@ printers, etc. However, due to the popularity of Windows, it is also supported on many other platforms including Linux. The most popular way of supporting SMB on Linux is via the software Samba. -[Samba](), and other -ways of serving Linux filesystems, via SMB should work fine with +### SMB as branch + +Using a SMB mount as a branch in mergerfs may result in problems +(permission errors) since it is not POSIX compliant. This setup is not +common and has not been extensively tested. + + +### SMB exporting mergerfs + +[Samba]() and other +ways of serving Linux filesystems via SMB should work fine with mergerfs. The services do not tend to use the same technologies which NFS uses and therefore don't have the same issues. There should not be -special settings required to use mergerfs with Samba. However, +special settings required export mergerfs with Samba. However, [CIFSD](https://en.wikipedia.org/wiki/CIFSD) and other programs have not been extensively tested. If you use mergerfs with CIFSD or other SMB servers please submit your experiences so these docs can be @@ -103,8 +140,8 @@ There are no known issues using sshfs with mergerfs. You may want to use the following arguments to improve performance but your millage may vary. -- `-o Ciphers=arcfour` -- `-o Compression=no` +* `-o Ciphers=arcfour` +* `-o Compression=no` More info can be found [here](https://ideatrash.net/2016/08/odds-and-ends-optimizing-sshfs-moving.html). diff --git a/mkdocs/mkdocs.yml b/mkdocs/mkdocs.yml index 18aac855..83e6c67c 100644 --- a/mkdocs/mkdocs.yml +++ b/mkdocs/mkdocs.yml @@ -21,6 +21,7 @@ theme: - navigation.tracking - optimize - search.share + - search.suggest - toc.follow palette: - media: "(prefers-color-scheme)" @@ -91,13 +92,13 @@ nav: - tooling.md - usage_patterns.md - FAQ: + - faq/why_isnt_it_working.md - faq/reliability_and_scalability.md - faq/usage_and_functionality.md - faq/configuration_and_policies.md - faq/compatibility_and_integration.md - faq/recommendations_and_warnings.md - faq/technical_behavior_and_limitations.md - - faq/why_isnt_it_working.md - faq/limit_drive_spinup.md - related_projects.md - media_and_publicity.md