Browse Source

Merge pull request #1447 from trapexit/branchcfg

Improve mount waiting + misc doc improvements
master
trapexit 4 hours ago
committed by GitHub
parent
commit
8ee998e8aa
No known key found for this signature in database GPG Key ID: B5690EEEBB952194
  1. 182
      mkdocs/docs/config/branches.md
  2. 48
      mkdocs/docs/config/branches_mount_timeout.md
  3. 86
      mkdocs/docs/config/functions_categories_policies.md
  4. 33
      mkdocs/docs/faq/why_isnt_it_working.md
  5. 130
      mkdocs/docs/media_and_publicity.md
  6. 4
      mkdocs/docs/quickstart.md
  7. 17
      mkdocs/docs/usage_patterns.md
  8. 4
      mkdocs/mkdocs.yml
  9. 3
      src/config.cpp
  10. 6
      src/config.hpp
  11. 14
      src/from_string.cpp
  12. 3
      src/from_string.hpp
  13. 6
      src/fs_mkdir.hpp
  14. 12
      src/fs_mktemp.cpp
  15. 37
      src/fs_mounts.cpp
  16. 21
      src/fs_mounts.hpp
  17. 91
      src/fs_wait_for_mount.cpp
  18. 3
      src/fs_wait_for_mount.hpp
  19. 11
      src/fuse_init.cpp
  20. 15
      src/fuse_link.cpp
  21. 2
      src/fuse_rename.cpp
  22. 47
      src/mergerfs.cpp
  23. 2
      src/option_parser.cpp
  24. 8
      src/to_string.cpp
  25. 3
      src/to_string.hpp
  26. 7
      src/tofrom_wrapper.hpp

182
mkdocs/docs/config/branches.md

@ -1,18 +1,20 @@
# branches
The 'branches' argument is a colon (':') delimited list of paths to be
The `branches` argument is a colon (':') delimited list of paths to be
pooled together. It does not matter if the paths are on the same or
different filesystems nor does it matter the filesystem type (within
reason). Used and available space will not be duplicated for paths on
the same filesystem and any features which aren't supported by the
underlying filesystem (such as file attributes or extended attributes)
will return the appropriate errors.
different filesystems nor does it matter [the filesystem
type](../faq/compatibility_and_integration.md/#what-filesystems-can-be-used-as-branches)
(within reason). Used and available space metrics will not be
duplicated for paths on the same filesystem and any features which
aren't supported by the underlying filesystem (such as file attributes
or extended attributes) will return the appropriate errors.
Branches currently have two options which can be set. A type which
impacts whether or not the branch is included in a policy calculation
and a individual minfreespace value. The values are set by prepending
an `=` at the end of a branch designation and using commas as
delimiters. Example: `/mnt/drive=RW,1234`
Branches currently have two options which can be set. A
[mode](#branch-mode) which impacts whether or not the branch is
included in a policy calculation and a individual
[minfreespace](#minfreespace) value. The values are set by
prepending an `=` at the end of a branch designation and using commas
as delimiters. Example: `/mnt/drive=RW,1234`
### branch mode
@ -29,7 +31,7 @@ delimiters. Example: `/mnt/drive=RW,1234`
### minfreespace
Same purpose and syntax as the [global option](minfreespace.md) but
specific to the branch. If not set the global value is used.
specific to the branch. Defaults to the global value.
### globbing
@ -43,8 +45,8 @@ apply the glob itself.**
# mergerfs /mnt/hdd\*:/mnt/ssd /media
```
The above line will use all mount points in /mnt prefixed with **hdd**
and **ssd**.
The above line will use all directories in /mnt prefixed with **hdd**
as well as **ssd**.
To have the pool mounted at boot or otherwise accessible from related
tools use `/etc/fstab`.
@ -54,4 +56,154 @@ tools use `/etc/fstab`.
/mnt/hdd*:/mnt/ssd /media mergerfs minfreespace=16G 0 0
```
**NOTE:** The globbing is done at mount or when updated using the runtime API. If a new directory is added matching the glob after the fact it will not be automatically included.
**NOTE:** The globbing is done at mount or when updated using the
runtime API. If a new directory is added matching the glob after the
fact it will not be automatically included.
A convenient way to configure branches is to use
[symlinks](../quickstart.md/#etcfstab-w-config-file).
## branch setup
mergerfs does not require any special setup of branch paths in order
to be used however here are some suggestions.
### layout
When you have a large collection of storage devices, types of storage,
and types of interconnects it can be useful to have a verbose naming
convention. My preference is to create a directory within `/mnt/` for
each major type of storage device and/or interconnect. `hdd` for
traditional hard drives; `ssd` for slower SATA based SSDs; `nvme` for
M.2 or U.2 NVME SSDs; `remote` for NFS, SMB, sshfs, etc.
The mount points within those directories are named after the size of
the storage and its serial number in the case of physical storage and
the hostname and remote filesystem for remote.
```
$ ls -lh /mnt/
total 16K
drwxr-xr-x 8 root root 4.0K Aug 18 2024 hdd
drwxr-xr-x 6 root root 4.0K Oct 8 2024 nvme
drwxr-xr-x 3 root root 4.0K Aug 24 2024 remote
drwxr-xr-x 3 root root 4.0K Jul 14 2024 ssd
$ ls -lh /mnt/hdd/
total 8K
d--------- 2 root root 4.0K Apr 14 15:58 10T-01234567
d--------- 2 root root 4.0K Apr 12 20:51 20T-12345678
$ ls -lh /mnt/nvme/
total 8K
d--------- 2 root root 4.0K Apr 14 16:00 1T-ABCDEFGH
d--------- 2 root root 4.0K Apr 14 23:24 1T-BCDEFGHI
$ ls -lh /mnt/remote/
total 8K
d--------- 2 root root 4.0K Apr 12 20:23 foo-sshfs
d--------- 2 root root 4.0K Apr 12 20:24 bar-nfs
# You can find the serial number of a drive using lsblk
$ lsblk -d -o NAME,PATH,SIZE,SERIAL
NAME PATH SIZE SERIAL
sda /dev/sda 9.1T 01234567
sdb /dev/sdb 18.2T 12345678
nvme0n1 /dev/nvme0n1 953.9G ABCDEFGH
nvme1n1 /dev/nvme1n1 953.9G BCDEFGHI
```
### permissions and mode
#### mount points
To ensure the directory is **only** used as a point to mount another
filesystem it is good to lock it down as much as possible. Be sure to
do this **before** mounting a filesystem to it.
```
$ sudo chown root:root /mnt/hdd/10T-XYZ
$ sudo chmod 0000 /mnt/hdd/10T-XYZ
$ sudo setfattr -n user.mergerfs.branch_mounts_here
$ sudo chattr +i /mnt/hdd/10T-XYZ
```
The extended attribute `user.mergerfs.branch_mounts_here` is used by
the [branches-mount-timeout](branches_mount_timeout.md) option to
recognize whether or not a mergerfs branch path points to the intended
filesystem.
The `chattr` is likely to only work on EXT{2,3,4} filesystems but will
restrict even `root` from modifying the directory or its content.
#### mounted filesystems
For those new to Linux, intending to be the primary individual logged
into the system, or simply want to simplify permissions it is
recommended to set the root of mounted filesystems like `/tmp/` is set
to. Owned by `root`, `ugo+rwx` and [sticky
bit](https://en.wikipedia.org/wiki/Sticky_bit) set.
This must be done **after** mounting the filesystem to the target
mount point.
```
$ sudo chown root:root /mnt/hdd/10T-SERIALNUM
$ sudo chmod 1777 /mnt/hdd/10T-SERIALNUM
$ sudo setfattr -n user.mergerfs.branch /mnt/hdd/10T-SERIALNUM
```
### formatting
While even less relevant to mergerfs than the details above the topic
does com up regularly with mergerfs users. When it comes to
partitioning and formatting it is suggested to keep things
simple. Most users of mergerfs will be using the whole drive capacity
and as such have a singular partition and filesystem on that
partition. Something many people don't realize is that the partition
is not necessary and actually can become problematic.
Rather than creating paritions just format the block device. Let's use
`/dev/sda` as an example.
```
$ lsblk -d -o NAME,PATH,SERIAL /dev/sda
NAME PATH SERIAL
sda /dev/sda 01234567
$ sudo mkfs.ext4 -m0 -L 01234567 /dev/sda
mke2fs 1.47.0 (5-Feb-2023)
Discarding device blocks: done
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: ede8bf96-9aff-464e-a4fb-a4ab8a197cd7
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
```
You can then use the serial number as the identifier in `fstab`.
```
# <file system> <mount point> <type> <options> <dump> <pass>
LABEL=01234567 /mnt/hdd/10TB-01234567 auto nofail 0 2
```
Benefits of doing it this way?
* One less thing (partitions) to configure and worry about.
* Guarantees alignment of blocks
* Makes it easier to use the drive with different enclosures. Some
SATA to USB adapters use 4K blocks while the drive itself is using
512b. If you create partitions with one block size and move the
drive to a controller that uses the other the offset on the device
where the filesystem starts will be misinterpreted. It is possible
to manually fix this but it isn't well documented and avoidable.

48
mkdocs/docs/config/branches_mount_timeout.md

@ -0,0 +1,48 @@
# branches-mount-timeout
Default: `0`
mergerfs is compatible with any path, whether it be a mount point or a
directory on your [root
filesystem.](https://en.wikipedia.org/wiki/Root_directory) It doesn’t
require branch paths to be mounted or to match a specific filesystem
at startup, and it operates effectively without needing details about
the intended filesystem. This flexibility eliminates the need to
manage mount ordering, which is particularly advantageous on modern
systems where filesystems are mounted asynchronously, resulting in
unpredictable mount sequences. While
[systemd](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html)
offers a way to enforce mount dependencies using the
[x-systemd.requires=PATH](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html#x-systemd.requires=)
option in /etc/fstab, applying this to every branch path is both
cumbersome and susceptible to errors.
`branches-mount-timeout` will cause mergerfs to wait for all branches
to become "mounted." The logic to determine "mounted" is as follows.
1. It is mounted if the branch directory has the extended attribute
`user.mergerfs.branch`
2. It is mounted if the branch directory contains a file named
`.mergerfs.branch`
3. It is **not** mounted if the branch directory has the extended
attribute `user.mergerfs.branch_mounts_here`
4. It is **not** mounted if the branch directory contains
a file named `.mergerfs.branch_mounts_here`
5. It is mounted if the `st_dev` value of the mergerfs mountpoint is
different from the branch path.
**NOTE:** If on a `systemd` based system and using `fstab` it is a
good idea to set the mount option
[x-systemd.mount-timeout](https://www.freedesktop.org/software/systemd/man/latest/systemd.mount.html#x-systemd.mount-timeout=)
to some value longer than `branches-mount-timeout`.
## branches-mount-timeout-fail
Default: `false`
When set to `true` mergerfs will fail entirely after
`branches-mount-timeout` expires without all branches being
mounted. If set to `false` it will simply ignore the mount status of
the branches and continue on. The details will be
[logged](../error_handling_and_logging.md).

86
mkdocs/docs/config/functions_categories_policies.md

@ -1,28 +1,16 @@
# functions, categories and policies
The POSIX filesystem API is made up of a number of
functions. **creat**, **stat**, **chown**, etc. For ease of
configuration in mergerfs, most of the core functions are grouped into
3 categories: **action**, **create**, and **search**. These functions
and categories can be assigned a policy which dictates which branch is
chosen when performing that function.
The POSIX filesystem API is made up of a number of functions. `creat`,
`stat`, `chown`, etc. For ease of configuration in mergerfs, most of
the core functions are grouped into 3 categories: `action`, `create`,
and `search`. These functions and categories can be assigned a policy
which dictates which branch is chosen when performing that
function.
Some functions, listed in the category `N/A` below, can not be
assigned the normal policies because they are directly related to a
file which has already been opened.
When using policies which are based on a branch's available space the
base path provided is used. Not the full path to the file in
question. Meaning that mounts in the branch won't be considered in the
space calculations. The reason is that it doesn't really work for
non-path preserving policies and can lead to non-obvious behaviors.
NOTE: While any policy can be assigned to a function or category,
some may not be very useful in practice. For instance: **rand**
(random) may be useful for file creation (create) but could lead to
very odd behavior if used for `chmod` if there were more than one copy
of the file.
## Functions and their Category classifications
@ -34,43 +22,61 @@ of the file.
| N/A | fchmod, fchown, futimens, ftruncate, fallocate, fgetattr, fsync, ioctl (files), read, readdir, release, statfs, write, copy_file_range |
In cases where something may be searched for (such as a path to clone)
**getattr** will usually be used.
`getattr` will usually be used.
## Policies
See below for [available policies and their descriptions](#policy-descriptions).
A policy is an algorithm designed to select one or more branches for a
function to operate on.
Any function in the `create` category will clone the relative path if
needed. Some other functions (`rename`,`link`,`ioctl`) have special
requirements or behaviors which you can read more about below.
Policies do not actually manage the filesystem or layout. They are
strictly responsible for deciding which files or branches will be
worked on in relation to the function being performed. Once the branch
is chosen other parts of the system does what is necessary to
accomplish the function. Such as the cloning of directories between
branches.
When using policies which are based on a branch's available space the
branch base path provided is used. Not the full path to the file or
directory in question. Meaning that mounts within the branch will not
be considered in the space calculations.
NOTE: While any policy can be assigned to a function or category, some
may not be very useful in practice. For instance: `rand` (random) may
be useful for file creation but could lead to very odd behavior if
used for `chmod` if there were more than one copy of the file. Unless
users find this flexibility useful it will likely be removed in the
future.
## Filtering
Most policies basically search branches and create a list of files / paths
for functions to work on. The policy is responsible for filtering and
sorting the branches. Filters include **minfreespace**, whether or not
a branch is mounted read-only, and the branch tagging
Most policies search branches and create a list of files / paths for
functions to work on. The policy is responsible for filtering and
sorting the branches. Filters include [minfreespace](minfreespace.md),
whether or not a branch is mounted read-only, and the branch mode
(RO,NC,RW). These filters are applied across most policies.
- No **search** function policies filter.
- All **action** function policies filter out branches which are
mounted **read-only** or tagged as **RO (read-only)**.
- All **create** function policies filter out branches which are
mounted **read-only**, tagged **RO (read-only)** or **NC (no
create)**, or has available space less than `minfreespace`.
- No `search` function policies filter.
- All `action` function policies filter out branches which are
mounted **read-only** or mode is **RO (read-only)**.
- All `create` function policies filter out branches which are
mounted **read-only**, mode **RO (read-only)** or **NC (no
create)**, or has available space less than
[minfreespace](minfreespace.md).
Policies may have their own additional filtering such as those that
require existing paths to be present.
If all branches are filtered an error will be returned. Typically
**EROFS** (read-only filesystem) or **ENOSPC** (no space left on
`EROFS` (read-only filesystem) or `ENOSPC` (no space left on
device) depending on the most recent reason for filtering a
branch. **ENOENT** will be returned if no eligible branch is found.
branch.
If **create**, **mkdir**, **mknod**, or **symlink** fail with `EROFS`
If `create`, `mkdir`, `mknod`, or `symlink` fail with `EROFS`
or other fundamental errors then mergerfs will mark any branch found
to be read-only as such (IE will set the mode `RO`) and will rerun the
policy and try again. This is mostly for `ext4` filesystems that can
@ -82,16 +88,12 @@ suddenly become read-only when it encounters an error.
Policies, as described below, are of two basic classifications. `path
preserving` and `non-path preserving`.
All policies which start with `ep` (**epff**, **eplfs**, **eplus**,
**epmfs**, **eprand**) are `path preserving`. `ep` stands for
`existing path`.
All policies which start with `ep` (`epff`, `eplfs`, `eplus`, `epmfs`,
`eprand`) are `path preserving`. `ep` stands for `existing path`.
A path preserving policy will only consider branches where the relative
path being accessed already exists.
When using non-path preserving policies paths will be cloned to target
branches as necessary.
With the `msp` or `most shared path` policies they are defined as
`path preserving` for the purpose of controlling `link` and `rename`'s
behaviors since `ignorepponrename` is available to disable that
@ -141,5 +143,5 @@ policies is not appropriate.
| Category | Policy |
| -------- | ------ |
| action | epall |
| create | epmfs |
| create | pfrd |
| search | ff |

33
mkdocs/docs/faq/why_isnt_it_working.md

@ -17,31 +17,30 @@ anywhere.
## Why are all my files ending up on 1 filesystem?!
Did you start with empty filesystems? Are you using an `existing path`
policy?
policy such as `epmfs`?
The default create policy is `epmfs`. That is a path preserving
algorithm. With such a policy for `mkdir` and `create` with a set of
empty filesystems it will select only 1 filesystem when the first
directory is created. Anything, files or directories, created in that
directory will be placed on the same branch because it is preserving
paths. That is the expected behavior.
If starting with a set of empty filesystems such a policy will select
only 1 filesystem when the first directory is created. Anything
created in that directory will be placed on the same branch because
that is the point of such a policy. That is the expected behavior.
This may catch new users off guard but this policy is the safest
policy to have as default as it will not change the general layout of
the underlying branches. If you do not care about path preservation
([most should
not](configuration_and_policies.md#how-can-i-ensure-files-are-collocated-on-the-same-branch))
and wish your files to be spread across all your filesystems change
`create.category` to `pfrd`, `rand`, `mfs` or a similarly non-path
restrictive [policy](../config/functions_categories_policies.md).
This may catch users off guard but the point of those `existing path`
policies is to preserve the directory layout and you need to consider
what that means one event at a time.
To "fix" this situation change the policy. Most users [have little
reason](configuration_and_policies.md#how-can-i-ensure-files-are-collocated-on-the-same-branch)
to use such a policy. See the [quick start guide](../quickstart.md)
and [FAQ](configuration_and_policies.md) for policy recommendations.
## Why do I get an "out of space" / "no space left on device" / ENOSPC error even though there appears to be lots of space available?
First make sure you've read the sections above about [policies, path
First make sure you've read the sections about [policies, path
preservation, branch
filtering,](../config/functions_categories_policies.md) and the
options `minfreespace`, [moveonenospc](../config/moveonenospc.md),
options [minfreespace](../config/minfreespace.md),
[moveonenospc](../config/moveonenospc.md),
[statfs](../config/statfs.md), and
[statfs_ignore](../config/statfs.md#statfs_ignore).

130
mkdocs/docs/media_and_publicity.md

@ -2,79 +2,79 @@
## Tutorials / Articles
- 2016-02-02 - [Linuxserver.io: The Perfect Media Server 2016](https://blog.linuxserver.io/2016/02/02/the-perfect-media-server-2016/)
- 2016-08-31 - [ZackReed.me: Mergerfs – another good option to pool your SnapRAID disks](https://zackreed.me/mergerfs-another-good-option-to-pool-your-snapraid-disks/)
- 2016-11-06 - [Linuxserver.io: Revisiting the HP ProLiant Gen8 G1610T Microserver](https://blog.linuxserver.io/2016/11/06/revisiting-the-hp-proliant-gen8-g1610t-microserver/)
- 2017-01-17 - [Setting up mergerfs on JBOD's (or a poor mans storage array)](http://corywestropp.com/develop/articles/setting-up-mergerfs/)
- 2017-06-24 - [Linuxserver.io: The Perfect Media Server 2017](https://blog.linuxserver.io/2017/06/24/the-perfect-media-server-2017/)
- 2018-02-19 - [Teknophiles: Disk Pooling in Linux with mergerFS](https://web.archive.org/web/20210324184857/https://www.teknophiles.com/2018/02/19/disk-pooling-in-linux-with-mergerfs/)
- 2018-02-20 - [Fortes.com: Using Rclone and MergerFS together across drives](https://fortes.com/2018/rclone-and-mergerfs/)
- 2019-02-10 - [Medium: Migrating from ZFS to MergerFS and SnapRAID at home](https://medium.com/@pascal.brokmeier/migrating-from-zfs-to-mergerfs-and-snapraid-at-home-89c45fd5db02)
- 2019-04-24 - [MichaelXander.com: DIY NAS with OMV, SnapRAID, MergerFS, and Disk Encryption](https://michaelxander.com/diy-nas/)
- 2019-07-16 - [Linuxserver.io: The Perfect Media Server - 2019 Edition](https://blog.linuxserver.io/2019/07/16/perfect-media-server-2019/)
- 2019-09-10 - [Rclone VFS and MergerFS Setup](https://docs.usbx.me/books/rclone/page/rclone-vfs-and-mergerfs-setup)
- 2019-12-20 - [NetworkShinobi.com: SnapRAID and MergerFS on OpenMediaVault](https://www.networkshinobi.com/snapraid-and-mergerfs-on-openmediavault/)
- 2020-01-14 - [Brandon Rozek's Blog](https://brandonrozek.com/blog/mergerfs/)
- 2020-02-14 - [SelfHostedHome.com: Combining Different Sized Drives with mergerfs and SnapRAID](https://selfhostedhome.com/combining-different-sized-drives-with-mergerfs-and-snapraid/)
- 2020-05-01 - [FedoraMagazine.org: Using mergerfs to increase your virtual storage](https://fedoramagazine.org/using-mergerfs-to-increase-your-virtual-storage/)
- 2020-08-20 - [Setting up Rclone, Mergerfs and Crontab for automated cloud storage](https://bytesized-hosting.com/pages/setting-up-rclone-mergerfs-and-crontab-for-automated-cloud-storage)
- 2020-11-22 - [Introducing… MergerFS – My FREE UNRAID alternative](https://supertechfreaks.com/introducing-mergerfs-free-unraid-alternative/)
- 2020-12-30 - [Perfect Media Server](https://perfectmediaserver.com) (a new site with docs fully fleshing out the 'Perfect Media Server' blog series)
- 2021-07-24 - [Building the Ultimate Linux Home Server - Part 1: Intro, MergerFS, and SnapRAID](https://blog.karaolidis.com/ultimate-home-server-part-1/)
- 2021-10-31 - [Better Home Storage: MergerFS + SnapRAID on OpenMediaVault](https://blog.sakuragawa.moe/better-home-storage-mergerfs-snapraid-on-openmediavault/)
- 2021-11-28 - [Linux Magazine: Come Together - Merging file systems for a simple NAS with MergerFS](https://www.linux-magazine.com/Issues/2022/254/MergerFS)
- 2022-06-04 - [MergerFS + SnapRaid Study](https://crashlaker.github.io/2022/06/04/mergerfs_+_snapraid_study.html)
- 2022-12-31 - [Merge Storages in CasaOS: A secret beta feature you know now](https://blog.casaos.io/blog/13.html)
- 2023-02-03 - [(MergerFS + SnapRAID) is the new RAID 5](https://thenomadcode.tech/mergerfs-snapraid-is-the-new-raid-5)
- 2024-02-07 - [Designing & Deploying MANS - A Hybrid NAS Approach with SnapRAID, MergerFS, and OpenZFS](https://blog.muffn.io/posts/part-3-mini-100tb-nas)
- 2024-03-11 - [Using MergerFS to combine multiple hard drives into one unified media storage](https://fullmetalbrackets.com/blog/two-drives-mergerfs/)
- 2024-12-20 - [Pooling multiple drives on my Raspberry Pi with mergerfs](https://sebi.io/posts/2024-12-20-pooling-multiple-drives-with-mergerfs/)
* 2016-02-02 - [Linuxserver.io: The Perfect Media Server 2016](https://blog.linuxserver.io/2016/02/02/the-perfect-media-server-2016/)
* 2016-08-31 - [ZackReed.me: Mergerfs – another good option to pool your SnapRAID disks](https://zackreed.me/mergerfs-another-good-option-to-pool-your-snapraid-disks/)
* 2016-11-06 - [Linuxserver.io: Revisiting the HP ProLiant Gen8 G1610T Microserver](https://blog.linuxserver.io/2016/11/06/revisiting-the-hp-proliant-gen8-g1610t-microserver/)
* 2017-01-17 - [Setting up mergerfs on JBOD's (or a poor mans storage array)](http://corywestropp.com/develop/articles/setting-up-mergerfs/)
* 2017-06-24 - [Linuxserver.io: The Perfect Media Server 2017](https://blog.linuxserver.io/2017/06/24/the-perfect-media-server-2017/)
* 2018-02-19 - [Teknophiles: Disk Pooling in Linux with mergerFS](https://web.archive.org/web/20210324184857/https://www.teknophiles.com/2018/02/19/disk-pooling-in-linux-with-mergerfs/)
* 2018-02-20 - [Fortes.com: Using Rclone and MergerFS together across drives](https://fortes.com/2018/rclone-and-mergerfs/)
* 2019-02-10 - [Medium: Migrating from ZFS to MergerFS and SnapRAID at home](https://medium.com/@pascal.brokmeier/migrating-from-zfs-to-mergerfs-and-snapraid-at-home-89c45fd5db02)
* 2019-04-24 - [MichaelXander.com: DIY NAS with OMV, SnapRAID, MergerFS, and Disk Encryption](https://michaelxander.com/diy-nas/)
* 2019-07-16 - [Linuxserver.io: The Perfect Media Server - 2019 Edition](https://blog.linuxserver.io/2019/07/16/perfect-media-server-2019/)
* 2019-09-10 - [Rclone VFS and MergerFS Setup](https://docs.usbx.me/books/rclone/page/rclone-vfs-and-mergerfs-setup)
* 2019-12-20 - [NetworkShinobi.com: SnapRAID and MergerFS on OpenMediaVault](https://www.networkshinobi.com/snapraid-and-mergerfs-on-openmediavault/)
* 2020-01-14 - [Brandon Rozek's Blog](https://brandonrozek.com/blog/mergerfs/)
* 2020-02-14 - [SelfHostedHome.com: Combining Different Sized Drives with mergerfs and SnapRAID](https://selfhostedhome.com/combining-different-sized-drives-with-mergerfs-and-snapraid/)
* 2020-05-01 - [FedoraMagazine.org: Using mergerfs to increase your virtual storage](https://fedoramagazine.org/using-mergerfs-to-increase-your-virtual-storage/)
* 2020-08-20 - [Setting up Rclone, Mergerfs and Crontab for automated cloud storage](https://bytesized-hosting.com/pages/setting-up-rclone-mergerfs-and-crontab-for-automated-cloud-storage)
* 2020-11-22 - [Introducing… MergerFS – My FREE UNRAID alternative](https://supertechfreaks.com/introducing-mergerfs-free-unraid-alternative/)
* 2020-12-30 - [Perfect Media Server](https://perfectmediaserver.com) (a new site with docs fully fleshing out the 'Perfect Media Server' blog series)
* 2021-07-24 - [Building the Ultimate Linux Home Server - Part 1: Intro, MergerFS, and SnapRAID](https://blog.karaolidis.com/ultimate-home-server-part-1/)
* 2021-10-31 - [Better Home Storage: MergerFS + SnapRAID on OpenMediaVault](https://blog.sakuragawa.moe/better-home-storage-mergerfs-snapraid-on-openmediavault/)
* 2021-11-28 - [Linux Magazine: Come Together - Merging file systems for a simple NAS with MergerFS](https://www.linux-magazine.com/Issues/2022/254/MergerFS)
* 2022-06-04 - [MergerFS + SnapRaid Study](https://crashlaker.github.io/2022/06/04/mergerfs_+_snapraid_study.html)
* 2022-12-31 - [Merge Storages in CasaOS: A secret beta feature you know now](https://blog.casaos.io/blog/13.html)
* 2023-02-03 - [(MergerFS + SnapRAID) is the new RAID 5](https://thenomadcode.tech/mergerfs-snapraid-is-the-new-raid-5)
* 2024-02-07 - [Designing & Deploying MANS - A Hybrid NAS Approach with SnapRAID, MergerFS, and OpenZFS](https://blog.muffn.io/posts/part-3-mini-100tb-nas)
* 2024-03-11 - [Using MergerFS to combine multiple hard drives into one unified media storage](https://fullmetalbrackets.com/blog/two-drives-mergerfs/)
* 2024-12-20 - [Pooling multiple drives on my Raspberry Pi with mergerfs](https://sebi.io/posts/2024-12-20-pooling-multiple-drives-with-mergerfs/)
## Videos
- 2017-06-23 - [Alex Kretzschmar: Part 1 - Perfect Media Server 2017 - Introduction](https://www.youtube.com/watch?v=L5MH8q3lmmk)
- 2017-06-24 - [Alex Kretzschmar: Part 2 - Perfect Media server 2017 - Installing Debian 9 Stretch](https://www.youtube.com/watch?v=YpVVYRN_L_A)
- 2017-06-24 - [Alex Kretzschmar: Part 3 - Perfect Media Server 2017 - Install MergerFS and setting up your drives](https://www.youtube.com/watch?v=tbCMfm-jJ5Y)
- 2017-06-24 - [Alex Kretzschmar: Part 4 - Perfect Media Server 2017 - Installing Docker](https://www.youtube.com/watch?v=WYI32kx4hPE)
- 2017-06-24 - [Alex Kretzschmar: Part 5 - Perfect Media Server 2017 - Installing and Automating SnapRAID](https://www.youtube.com/watch?v=Ir5ZsUIbHXA)
- 2017-06-24 - [Alex Kretzschmar: Part 6 - Perfect Media Server 2017 -Turning your server into a NAS with Samba and NFS](https://www.youtube.com/watch?v=1hVdWq758ZQ)
- 2017-06-24 - [Alex Kretzschmar: Part 7 - Perfect Media Server 2017 - Managing your apps with docker-compose](https://www.youtube.com/watch?v=aI2rdw7_AmE)
- 2017-06-24 - [Alex Kretzschmar: Part 8 - Perfect Media Server 2017 - Manging your server using a web UI (Cockpit and Portainer)](https://www.youtube.com/watch?v=aLyTWdzDiCg)
- 2018-04-24 - [ElectronicsWizardy: How to setup a Linux Fileserver with Snapraid and Mergerfs Re-Export](https://www.youtube.com/watch?v=D2Klx-X7pFo)
- 2019-03-01 - [Snapraid and Unionfs: Advanced Array Options on Openmediavault (Better than ZFS and Unraid)](https://www.youtube.com/watch?v=FYkdPyCt5FU)
- 2019-03-22 - [Gamexplicit: The Perfect Plex Media Server 2019! Part 1 (Hardware)](https://www.youtube.com/watch?v=rJIRPhM2WcE)
- 2019-05-13 - [Gamexplicit: The Perfect Plex Media Server 2019! Part 2 (Ubuntu Server 18.04.2 LTS)](https://www.youtube.com/watch?v=aLyTWdzDiCg)
- 2019-05-20 - [Gamexplicit: The Perfect Plex Media Server 2019! Part 3 (SnapRaid, Samba, Plex)](https://www.youtube.com/watch?v=uW5y43XC-BI)
- 2020-08-23 - [Installing OpenMediaVault and SnapRAID and UnionFS (mergerfs)](https://www.youtube.com/watch?v=nDvzXM8UjAI)
- 2021-06-07 - [I ditched TrueNAS for MergerFS - Chia Plot Storage](https://www.youtube.com/watch?v=tpqFywkbZa4)
- 2021-08-03 - [Install and configure mergerfs to merge more than one folder in the same place](https://www.youtube.com/watch?v=69zcqEy1674)
- 2021-08-10 - [Instalando e configurando mergerfs para unir mais de uma pasta no mesmo lugar](https://www.youtube.com/watch?v=-RLxbBNBWhU)
- 2021-08-13 - [Let's Convert an Old Laptop to a NAS - What you should do](https://www.youtube.com/watch?v=F1v-TSbOymI)
- 2021-08-17 - [How to Combine Multiple Disks as One by using MergerFS | Ubuntu 20.04 LTS](https://www.youtube.com/watch?v=9e46pz5Seo4)
- 2021-08-20 - [Vamos converter um Laptop antigo para um NAS – O que você deve fazer](https://www.youtube.com/watch?v=q8EK9vWCRTc)
- 2021-10-29 - [Unlimited space for your Plex using Rclone to connect to your Cloud](https://www.youtube.com/watch?v=ghGconyrF3M)
- 2022-12-01 - [Make Your Home Server Go FAST! SSD Caching, 10Gbit Networking, etc.](https://www.youtube.com/watch?v=eRfqC_q3lkM&t=784s)
- 2022-12-07 - [Best RAID for mixed drive sizes. Unraid vs BTRFS vs Snapraid+Mergerfs vs Storage spaces.](https://www.youtube.com/watch?v=NQJkTiLXfgs)
- 2023-02-21 - [MergerFS + SnapRAID : Forget about RAID 5 in your Home Server !](https://www.youtube.com/watch?v=tX5MA-c6Qq4)
- 2023-06-26 - [How to install and setup MergerFS](https://www.youtube.com/watch?v=n7piuhTXeG4)
- 2023-07-31 - [How to recover a dead drive using Snapraid](https://www.youtube.com/watch?v=fmuiRLPcuJE)
- 2024-01-05 - [OpenMediaVault MergerFS Tutorial (Portuguese)](https://www.youtube.com/watch?v=V6Yw86dRUPQ)
- 2024-02-19 - [Setup and Install MergerFS and SnapRAID (Part 1)](https://noted.lol/mergerfs-and-snapraid-setup-1/)
- 2024-02-22 - [Setup and Install MergerFS and SnapRAID (Part 2)](https://noted.lol/mergerfs-and-snapraid-setup-part-2/)
- 2024-11-15 - [Meu servidor NAS - Parte 18: Recuperando um HD, recuperando o MergerFS e os próximos passos do NAS!](https://www.youtube.com/watch?v=5fy98kPzE3s)
* 2017-06-23 - [Alex Kretzschmar: Part 1 - Perfect Media Server 2017 - Introduction](https://www.youtube.com/watch?v=L5MH8q3lmmk)
* 2017-06-24 - [Alex Kretzschmar: Part 2 - Perfect Media server 2017 - Installing Debian 9 Stretch](https://www.youtube.com/watch?v=YpVVYRN_L_A)
* 2017-06-24 - [Alex Kretzschmar: Part 3 - Perfect Media Server 2017 - Install MergerFS and setting up your drives](https://www.youtube.com/watch?v=tbCMfm-jJ5Y)
* 2017-06-24 - [Alex Kretzschmar: Part 4 - Perfect Media Server 2017 - Installing Docker](https://www.youtube.com/watch?v=WYI32kx4hPE)
* 2017-06-24 - [Alex Kretzschmar: Part 5 - Perfect Media Server 2017 - Installing and Automating SnapRAID](https://www.youtube.com/watch?v=Ir5ZsUIbHXA)
* 2017-06-24 - [Alex Kretzschmar: Part 6 - Perfect Media Server 2017 -Turning your server into a NAS with Samba and NFS](https://www.youtube.com/watch?v=1hVdWq758ZQ)
* 2017-06-24 - [Alex Kretzschmar: Part 7 - Perfect Media Server 2017 - Managing your apps with docker-compose](https://www.youtube.com/watch?v=aI2rdw7_AmE)
* 2017-06-24 - [Alex Kretzschmar: Part 8 - Perfect Media Server 2017 - Manging your server using a web UI (Cockpit and Portainer)](https://www.youtube.com/watch?v=aLyTWdzDiCg)
* 2018-04-24 - [ElectronicsWizardy: How to setup a Linux Fileserver with Snapraid and Mergerfs Re-Export](https://www.youtube.com/watch?v=D2Klx-X7pFo)
* 2019-03-01 - [Snapraid and Unionfs: Advanced Array Options on Openmediavault (Better than ZFS and Unraid)](https://www.youtube.com/watch?v=FYkdPyCt5FU)
* 2019-03-22 - [Gamexplicit: The Perfect Plex Media Server 2019! Part 1 (Hardware)](https://www.youtube.com/watch?v=rJIRPhM2WcE)
* 2019-05-13 - [Gamexplicit: The Perfect Plex Media Server 2019! Part 2 (Ubuntu Server 18.04.2 LTS)](https://www.youtube.com/watch?v=aLyTWdzDiCg)
* 2019-05-20 - [Gamexplicit: The Perfect Plex Media Server 2019! Part 3 (SnapRaid, Samba, Plex)](https://www.youtube.com/watch?v=uW5y43XC-BI)
* 2020-08-23 - [Installing OpenMediaVault and SnapRAID and UnionFS (mergerfs)](https://www.youtube.com/watch?v=nDvzXM8UjAI)
* 2021-06-07 - [I ditched TrueNAS for MergerFS - Chia Plot Storage](https://www.youtube.com/watch?v=tpqFywkbZa4)
* 2021-08-03 - [Install and configure mergerfs to merge more than one folder in the same place](https://www.youtube.com/watch?v=69zcqEy1674)
* 2021-08-10 - [Instalando e configurando mergerfs para unir mais de uma pasta no mesmo lugar](https://www.youtube.com/watch?v=-RLxbBNBWhU)
* 2021-08-13 - [Let's Convert an Old Laptop to a NAS - What you should do](https://www.youtube.com/watch?v=F1v-TSbOymI)
* 2021-08-17 - [How to Combine Multiple Disks as One by using MergerFS | Ubuntu 20.04 LTS](https://www.youtube.com/watch?v=9e46pz5Seo4)
* 2021-08-20 - [Vamos converter um Laptop antigo para um NAS – O que você deve fazer](https://www.youtube.com/watch?v=q8EK9vWCRTc)
* 2021-10-29 - [Unlimited space for your Plex using Rclone to connect to your Cloud](https://www.youtube.com/watch?v=ghGconyrF3M)
* 2022-12-01 - [Make Your Home Server Go FAST! SSD Caching, 10Gbit Networking, etc.](https://www.youtube.com/watch?v=eRfqC_q3lkM&t=784s)
* 2022-12-07 - [Best RAID for mixed drive sizes. Unraid vs BTRFS vs Snapraid+Mergerfs vs Storage spaces.](https://www.youtube.com/watch?v=NQJkTiLXfgs)
* 2023-02-21 - [MergerFS + SnapRAID : Forget about RAID 5 in your Home Server !](https://www.youtube.com/watch?v=tX5MA-c6Qq4)
* 2023-06-26 - [How to install and setup MergerFS](https://www.youtube.com/watch?v=n7piuhTXeG4)
* 2023-07-31 - [How to recover a dead drive using Snapraid](https://www.youtube.com/watch?v=fmuiRLPcuJE)
* 2024-01-05 - [OpenMediaVault MergerFS Tutorial (Portuguese)](https://www.youtube.com/watch?v=V6Yw86dRUPQ)
* 2024-02-19 - [Setup and Install MergerFS and SnapRAID (Part 1)](https://noted.lol/mergerfs-and-snapraid-setup-1/)
* 2024-02-22 - [Setup and Install MergerFS and SnapRAID (Part 2)](https://noted.lol/mergerfs-and-snapraid-setup-part-2/)
* 2024-11-15 - [Meu servidor NAS - Parte 18: Recuperando um HD, recuperando o MergerFS e os próximos passos do NAS!](https://www.youtube.com/watch?v=5fy98kPzE3s)
* 2025-04-23 - [How to build the Perfect Media Server | Part 1 - The Tech Stack | mergerfs, SnapRAID, and docker.](https://www.youtube.com/watch?v=Yt67zz9p0FU)
## Podcasts
- 2019-11-04 - [Jupiter Extras: A Chat with mergerfs Developer Antonio Musumeci | Jupiter Extras 28](https://www.youtube.com/watch?v=VmJUAyyhSPk)
- 2019-11-07 - [Jupiter Broadcasting: ZFS Isn’t the Only Option | Self-Hosted 5](https://www.youtube.com/watch?v=JEW7UuKhMJ8)
- 2023-10-08 - [Self Hosted Episode 105 - Sleeper Storage Technology](https://selfhosted.show/105)
* 2019-11-04 - [Jupiter Extras: A Chat with mergerfs Developer Antonio Musumeci | Jupiter Extras 28](https://www.youtube.com/watch?v=VmJUAyyhSPk)
* 2019-11-07 - [Jupiter Broadcasting: ZFS Isn’t the Only Option | Self-Hosted 5](https://www.youtube.com/watch?v=JEW7UuKhMJ8)
* 2023-10-08 - [Self Hosted Episode 105 - Sleeper Storage Technology](https://selfhosted.show/105)
## Social Media
- [Reddit](https://www.reddit.com/search/?q=mergerfs&sort=new)
- [X](https://x.com/search?q=mergerfs&src=spelling_expansion_revert_click&f=live)
- [YouTube](https://www.youtube.com/results?search_query=mergerfs&sp=CAI%253D)
- [ServeTheHome Forum](https://forums.servethehome.com/index.php?search/3105813/&q=mergerfs&o=date)
* [Reddit](https://www.reddit.com/search/?q=mergerfs&sort=new)
* [X](https://x.com/search?q=mergerfs&src=spelling_expansion_revert_click&f=live)
* [YouTube](https://www.youtube.com/results?search_query=mergerfs&sp=CAI%253D)
* [ServeTheHome Forum](https://forums.servethehome.com/index.php?search/3105813/&q=mergerfs&o=date)

4
mkdocs/docs/quickstart.md

@ -59,6 +59,10 @@ IO.)
## Usage
For some suggestions on branch setup see [the page on
branches](config/branches.md#branch-setup).
### Command Line
```

17
mkdocs/docs/usage_patterns.md

@ -4,14 +4,15 @@
Some storage technologies support what is called "tiered" caching. The
placing of smaller, faster storage as a transparent cache to larger,
slower storage. NVMe, SSD, Optane in front of traditional HDDs for
slower storage. NVMe, SSD, or Optane in front of traditional HDDs for
instance.
mergerfs does not natively support any sort of tiered caching. Most
users have no use for such a feature and its inclusion would
complicate the code as it exists today. However, there are a few
situations where a cache filesystem could help with a typical mergerfs
setup.
mergerfs does not natively support any sort of tiered caching
currently. The truth is for many users a cache would have little or no
advantage over reading and writing directly. They would be
bottlenecked by their network, internet connection, or limited size of
the cache. However, there are a few situations where a tiered cache
setup could help.
1. Fast network, slow filesystems, many readers: You've a 10+Gbps
network with many readers and your regular filesystems can't keep
@ -34,7 +35,7 @@ dm-cache but there is another solution which requires only mergerfs, a
script to move files around, and a cron job to run said script.
* Create two mergerfs pools. One which includes just the **slow**
branches and one which has both the **fast** branches
branches and one which has **both** the **fast** branches
(SSD,NVME,etc.) and **slow** branches. The **base** pool and the
**cache** pool.
* The **cache** pool should have the cache branches listed first in
@ -43,7 +44,7 @@ script to move files around, and a cron job to run said script.
probably be `ff`, `lus`, or `lfs`. The latter two under the
assumption that the cache filesystem(s) are far smaller than the
backing filesystems.
* You can also set the **slow** filesystems mode to `NC` which would
* You can also set the **slow** branches' mode to `NC` which would
give you the ability to use other `create` policies though that'd
mean if the cache filesystems fill you'd get "out of space"
errors. This however may be good as it would indicate the script

4
mkdocs/mkdocs.yml

@ -5,6 +5,9 @@ repo_name: mergerfs
repo_url: https://github.com/trapexit/mergerfs
edit_uri: edit/master/mkdocs/docs/
docs_dir: docs
plugins:
- search
- tags
theme:
name: material
logo: logo.png
@ -62,6 +65,7 @@ nav:
- config/options.md
- config/deprecated_options.md
- config/branches.md
- config/branches_mount_timeout.md
- config/functions_categories_policies.md
- config/minfreespace.md
- config/func_readdir.md

3
src/config.cpp

@ -56,6 +56,7 @@ namespace l
{
IFERT("async_read");
IFERT("branches-mount-timeout");
IFERT("branches-mount-timeout-fail");
IFERT("cache.symlinks");
IFERT("cache.writeback");
IFERT("direct-io-allow-mmap");
@ -85,6 +86,7 @@ Config::Config()
minfreespace(MINFREESPACE_DEFAULT),
branches(minfreespace),
branches_mount_timeout(0),
branches_mount_timeout_fail(false),
cache_attr(1),
cache_entry(1),
cache_files(CacheFiles::ENUM::OFF),
@ -139,6 +141,7 @@ Config::Config()
_map["auto_cache"] = &auto_cache;
_map["branches"] = &branches;
_map["branches-mount-timeout"] = &branches_mount_timeout;
_map["branches-mount-timeout-fail"] = &branches_mount_timeout_fail;
_map["cache.attr"] = &cache_attr;
_map["cache.entry"] = &cache_entry;
_map["cache.files"] = &cache_files;

6
src/config.hpp

@ -41,6 +41,8 @@
#include "rwlock.hpp"
#include "tofrom_wrapper.hpp"
#include "ghc/filesystem.hpp"
#include "fuse.h"
#include <cstdint>
@ -55,6 +57,7 @@ typedef ToFromWrapper<bool> ConfigBOOL;
typedef ToFromWrapper<uint64_t> ConfigUINT64;
typedef ToFromWrapper<int> ConfigINT;
typedef ToFromWrapper<std::string> ConfigSTR;
typedef ToFromWrapper<ghc::filesystem::path> ConfigPath;
typedef std::map<std::string,ToFromString*> Str2TFStrMap;
extern const std::string CONTROLFILE;
@ -108,6 +111,7 @@ public:
ConfigUINT64 minfreespace;
Branches branches;
ConfigUINT64 branches_mount_timeout;
ConfigBOOL branches_mount_timeout_fail;
ConfigUINT64 cache_attr;
ConfigUINT64 cache_entry;
CacheFiles cache_files;
@ -133,7 +137,7 @@ public:
ConfigBOOL link_cow;
LinkEXDEV link_exdev;
LogMetrics log_metrics;
ConfigSTR mountpoint;
ConfigPath mountpoint;
MoveOnENOSPC moveonenospc;
NFSOpenHack nfsopenhack;
ConfigBOOL nullrw;

14
src/from_string.cpp

@ -16,12 +16,11 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "from_string.hpp"
#include "ef.hpp"
#include "errno.hpp"
#include <cstdint>
#include <string>
#include <stdlib.h>
@ -128,4 +127,13 @@ namespace str
{
return -EINVAL;
}
int
from(const std::string &value_,
fs::Path *path_)
{
*path_ = value_;
return 0;
}
}

3
src/from_string.hpp

@ -18,6 +18,8 @@
#pragma once
#include "fs_path.hpp"
#include <cstdint>
#include <string>
@ -29,4 +31,5 @@ namespace str
int from(const std::string &, uint64_t *);
int from(const std::string &, std::string *);
int from(const std::string &, const std::string *);
int from(const std::string &, fs::Path *);
}

6
src/fs_mkdir.hpp

@ -18,7 +18,7 @@
#pragma once
#include "ghc/filesystem.hpp"
#include "fs_path.hpp"
#include <string>
@ -49,8 +49,8 @@ namespace fs
static
inline
int
mkdir(const ghc::filesystem::path &path_,
const mode_t mode_)
mkdir(const fs::Path &path_,
const mode_t mode_)
{
return fs::mkdir(path_.c_str(),mode_);
}

12
src/fs_mktemp.cpp

@ -30,7 +30,7 @@
#define PAD_LEN 16
#define MAX_ATTEMPTS 3
static char const CHARS[] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
static char const CHARS[] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
static size_t const CHARS_SIZE = (sizeof(CHARS) - 1);
@ -57,8 +57,8 @@ namespace l
namespace fs
{
std::tuple<int,std::string>
mktemp_in_dir(std::string const dirpath_,
int const flags_)
mktemp_in_dir(const std::string dirpath_,
const int flags_)
{
int fd;
int count;
@ -85,10 +85,10 @@ namespace fs
}
std::tuple<int,std::string>
mktemp(std::string const filepath_,
int const flags_)
mktemp(const std::string filepath_,
const int flags_)
{
ghc::filesystem::path filepath{filepath_};
fs::Path filepath{filepath_};
return fs::mktemp_in_dir(filepath.parent_path(),flags_);
}

37
src/fs_mounts.cpp

@ -0,0 +1,37 @@
#include "fs_mounts.hpp"
#include <cstdio>
#include <mntent.h>
#ifdef __linux__
void
fs::mounts(fs::MountVec &mounts_)
{
FILE *f;
f = setmntent("/proc/mounts","r");
if(f == NULL)
return;
struct mntent *entry;
while((entry = getmntent(f)) != NULL)
{
fs::Mount m;
m.dir = entry->mnt_dir;
m.fsname = entry->mnt_fsname;
m.type = entry->mnt_type;
m.opts = entry->mnt_opts;
mounts_.emplace_back(std::move(m));
}
endmntent(f);
}
#else
void
fs::mounts(fs::MountVec &mounts_)
{
}
#endif

21
src/fs_mounts.hpp

@ -0,0 +1,21 @@
#pragma once
#include "fs_path.hpp"
#include <vector>
namespace fs
{
struct Mount
{
fs::Path dir;
std::string fsname;
std::string type;
std::string opts;
};
typedef std::vector<fs::Mount> MountVec;
void mounts(fs::MountVec &mounts);
}

91
src/fs_wait_for_mount.cpp

@ -19,6 +19,10 @@
#include "fs_wait_for_mount.hpp"
#include "syslog.hpp"
#include "fs_exists.hpp"
#include "fs_lstat.hpp"
#include "fs_lgetxattr.hpp"
#include <functional>
#include <thread>
#include <unordered_set>
@ -44,6 +48,40 @@ namespace std
};
}
static
bool
_branch_is_mounted(const struct stat &src_st_,
const fs::Path &branch_path_)
{
int rv;
struct stat st;
fs::Path filepath;
rv = fs::lgetxattr(branch_path_,"user.mergerfs.branch",NULL,0);
if(rv != -1)
return true;
filepath = branch_path_ / ".mergerfs.branch";
rv = fs::exists(filepath);
if(rv)
return true;
rv = fs::lgetxattr(branch_path_,"user.mergerfs.branch_mounts_here",NULL,0);
if(rv != -1)
return false;
filepath = branch_path_ / ".mergerfs.branch_mounts_here";
rv = fs::exists(filepath);
if(rv)
return false;
rv = fs::lstat(branch_path_,&st);
if(rv == 0)
return (st.st_dev != src_st_.st_dev);
return false;
}
static
void
_check_mounted(const struct stat &src_st_,
@ -56,30 +94,23 @@ _check_mounted(const struct stat &src_st_,
for(auto const &tgt_path : tgt_paths_)
{
int rv;
struct stat tgt_st;
bool mounted;
rv = fs::stat(tgt_path,&tgt_st);
if(rv == 0)
{
if(tgt_st.st_dev != src_st_.st_dev)
successes.push_back(tgt_path);
else
failures.push_back(tgt_path);
}
mounted = ::_branch_is_mounted(src_st_,tgt_path);
if(mounted)
successes.push_back(tgt_path);
else
{
failures.push_back(tgt_path);
}
failures.push_back(tgt_path);
}
}
static
void
int
_wait_for_mount(const struct stat &src_st_,
const fs::PathVector &tgt_paths_,
const std::chrono::milliseconds &timeout_)
{
bool first_loop;
fs::PathVector successes;
fs::PathVector failures;
std::unordered_set<fs::Path> tgt_paths;
@ -90,6 +121,7 @@ _wait_for_mount(const struct stat &src_st_,
now = std::chrono::steady_clock::now();
deadline = now + timeout_;
first_loop = true;
while(true)
{
if(tgt_paths.empty())
@ -100,10 +132,17 @@ _wait_for_mount(const struct stat &src_st_,
successes.clear();
failures.clear();
::_check_mounted(src_st_,tgt_paths,&successes,&failures);
for(auto const &path : successes)
for(const auto &path : successes)
{
tgt_paths.erase(path);
syslog_info("%s is mounted",path.string().c_str());
syslog_info("%s is mounted",path.c_str());
}
if(first_loop)
{
for(const auto &path : failures)
syslog_notice("%s is not mounted, waiting",path.c_str());
first_loop = false;
}
std::this_thread::sleep_for(SLEEP_DURATION);
@ -111,26 +150,24 @@ _wait_for_mount(const struct stat &src_st_,
}
for(auto const &path : failures)
syslog_notice("%s not mounted within timeout",path.string().c_str());
if(!failures.empty())
syslog_warning("Continuing to mount mergerfs despite %u branches not "
"being different from the mountpoint filesystem",
failures.size());
syslog_notice("%s not mounted within timeout",path.c_str());
return failures.size();
}
void
int
fs::wait_for_mount(const fs::Path &src_path_,
const fs::PathVector &tgt_paths_,
const std::chrono::milliseconds &timeout_)
{
int rv;
struct stat src_st;
struct stat src_st = {0};
rv = fs::stat(src_path_,&src_st);
if(rv == -1)
return syslog_error("Error stat'ing mount path: %s (%s)",
src_path_.c_str(),
strerror(errno));
syslog_error("Error stat'ing mount path: %s (%s)",
src_path_.c_str(),
strerror(errno));
::_wait_for_mount(src_st,tgt_paths_,timeout_);
return ::_wait_for_mount(src_st,tgt_paths_,timeout_);
}

3
src/fs_wait_for_mount.hpp

@ -1,6 +1,5 @@
/*
ISC License
Copyright (c) 2023, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
@ -29,7 +28,7 @@
namespace fs
{
void
int
wait_for_mount(const fs::Path &srcpath,
const fs::PathVector &tgtpaths,
const std::chrono::milliseconds &timeout);

11
src/fuse_init.cpp

@ -19,12 +19,13 @@
#include "fs_readahead.hpp"
#include "syslog.hpp"
#include "fs_path.hpp"
#include "fs_exists.hpp"
#include "fmt/core.h"
#include "fuse.h"
#include "ghc/filesystem.hpp"
#include <thread>
#include <algorithm>
@ -82,7 +83,7 @@ namespace l
std::fstream f;
uint64_t max_pages_limit;
if(ghc::filesystem::exists(MAX_PAGES_LIMIT_FILEPATH))
if(fs::exists(MAX_PAGES_LIMIT_FILEPATH))
{
if(cfg_->fuse_msg_size > MAX_FUSE_MSG_SIZE)
syslog_info("fuse_msg_size > %u: setting it to %u",
@ -141,8 +142,8 @@ namespace l
static
void
readahead(const std::string path_,
const int readahead_)
readahead(const fs::Path path_,
const int readahead_)
{
int rv;

15
src/fuse_link.cpp

@ -32,7 +32,6 @@
using std::string;
using std::vector;
namespace gfs = ghc::filesystem;
namespace error
{
@ -230,8 +229,8 @@ namespace l
fuse_timeouts_t *timeouts_)
{
int rv;
gfs::path target(oldpath_);
gfs::path linkpath(newpath_);
fs::Path target(oldpath_);
fs::Path linkpath(newpath_);
target = target.lexically_relative(linkpath.parent_path());
@ -278,11 +277,11 @@ namespace l
static
int
link_exdev_abs_pool_symlink(const std::string &mount_,
const char *oldpath_,
const char *newpath_,
struct stat *st_,
fuse_timeouts_t *timeouts_)
link_exdev_abs_pool_symlink(const fs::Path mount_,
const char *oldpath_,
const char *newpath_,
struct stat *st_,
fuse_timeouts_t *timeouts_)
{
int rv;
StrVec basepaths;

2
src/fuse_rename.cpp

@ -315,7 +315,7 @@ namespace l
int
rename_exdev_abs_symlink(const Policy::Action &actionPolicy_,
const Branches::CPtr &branches_,
const std::string &mount_,
const gfs::path &mount_,
const gfs::path &oldfusepath_,
const gfs::path &newfusepath_)
{

47
src/mergerfs.cpp

@ -165,26 +165,51 @@ namespace l
}
static
void
bool
wait_for_mount(const Config::Read &cfg_)
{
int failures;
fs::PathVector paths;
std::chrono::milliseconds timeout;
paths = cfg_->branches->to_paths();
syslog_info("Waiting %u seconds for branches to mount",
(uint64_t)cfg_->branches_mount_timeout);
syslog_info("Waiting %u seconds for %zu branches to mount",
(uint64_t)cfg_->branches_mount_timeout,
paths.size());
timeout = std::chrono::milliseconds(cfg_->branches_mount_timeout * 1000);
fs::wait_for_mount((std::string)cfg_->mountpoint,
paths,
timeout);
failures = fs::wait_for_mount(cfg_->mountpoint,
paths,
timeout);
if(failures)
{
if(cfg_->branches_mount_timeout_fail)
{
syslog_error("%d of %zu branches were not mounted"
" within the timeout of %zus. Exiting",
failures,
paths.size(),
(uint64_t)cfg_->branches_mount_timeout);
return true;
}
syslog_warning("Continuing to mount mergerfs despite %d branches not "
"being different from the mountpoint filesystem",
failures);
}
else
{
syslog_info("All %zd branches are mounted",
paths.size());
}
return false;
}
static
void
lazy_umount(const std::string target_)
lazy_umount(const fs::Path target_)
{
int rv;
@ -276,7 +301,13 @@ namespace l
}
if(cfg->branches_mount_timeout > 0)
l::wait_for_mount(cfg);
{
bool failure;
failure = l::wait_for_mount(cfg);
if(failure)
return 1;
}
l::setup_resources(cfg->scheduling_priority);
l::setup_signal_handlers();

2
src/option_parser.cpp

@ -421,7 +421,7 @@ check_for_mount_loop(Config::Write &cfg_,
fs::PathVector branches;
std::error_code ec;
mount = (std::string)cfg_->mountpoint;
mount = *cfg_->mountpoint;
branches = cfg_->branches->to_paths();
for(const auto &branch : branches)
{

8
src/to_string.cpp

@ -16,6 +16,8 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include "to_string.hpp"
#include <cstdint>
#include <string>
@ -55,4 +57,10 @@ namespace str
{
return s_;
}
std::string
to(const fs::Path &path_)
{
return path_.string();
}
}

3
src/to_string.hpp

@ -18,6 +18,8 @@
#pragma once
#include "fs_path.hpp"
#include <cstdint>
#include <string>
@ -28,4 +30,5 @@ namespace str
std::string to(const int);
std::string to(const uint64_t);
std::string to(const std::string&);
std::string to(const fs::Path&);
}

7
src/tofrom_wrapper.hpp

@ -64,6 +64,13 @@ public:
return _data;
}
const
T&
operator*() const
{
return _data;
}
T*
operator->()
{

Loading…
Cancel
Save