From 579690f411c68a65c052bd7281735101a96ba9c4 Mon Sep 17 00:00:00 2001 From: Antonio SJ Musumeci Date: Thu, 27 Mar 2025 11:08:00 -0500 Subject: [PATCH] Add links to options docs page to individual pages --- mkdocs/docs/config/deprecated_options.md | 2 +- ...es.md => functions_categories_policies.md} | 0 mkdocs/docs/config/options.md | 180 +++++++++--------- mkdocs/docs/faq/why_isnt_it_working.md | 16 +- mkdocs/docs/index.md | 2 +- mkdocs/docs/intro_to_filesystems.md | 47 +++++ mkdocs/mkdocs.yml | 3 +- 7 files changed, 154 insertions(+), 96 deletions(-) rename mkdocs/docs/config/{functions_categories_and_policies.md => functions_categories_policies.md} (100%) create mode 100644 mkdocs/docs/intro_to_filesystems.md diff --git a/mkdocs/docs/config/deprecated_options.md b/mkdocs/docs/config/deprecated_options.md index 4b2b2ccd..69b6b872 100644 --- a/mkdocs/docs/config/deprecated_options.md +++ b/mkdocs/docs/config/deprecated_options.md @@ -1,7 +1,7 @@ # Deprecated Options These are old, deprecated options which may no longer have any -function or have been replaced. +function or have been replaced. **They should not be used.** * **direct_io**: Bypass page cache. Use `cache.files=off` instead. diff --git a/mkdocs/docs/config/functions_categories_and_policies.md b/mkdocs/docs/config/functions_categories_policies.md similarity index 100% rename from mkdocs/docs/config/functions_categories_and_policies.md rename to mkdocs/docs/config/functions_categories_policies.md diff --git a/mkdocs/docs/config/options.md b/mkdocs/docs/config/options.md index 7bdeb4f9..170c1d74 100644 --- a/mkdocs/docs/config/options.md +++ b/mkdocs/docs/config/options.md @@ -20,19 +20,22 @@ These options are the same regardless of whether you use them with the - **config**: Path to a config file. Same arguments as below in key=val / ini style format. -- **branches**: Colon delimited list of branches. -- **minfreespace=SIZE**: The minimum space value used for creation - policies. Can be overridden by branch specific option. Understands - 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte - respectively. (default: 4G) +- **[branches](branches.md)**: Colon delimited list of branches. Used + primarily in config file. +- **minfreespace=SIZE**: The minimum available space of a branch + necessary to be considered for a create + [policy](functions_categories_policies.md). This is a default value + applied to all branches and can be overwritten when configuring + [branches](branches.md). Understands 'K', 'M', and 'G' to represent + kilobyte, megabyte, and gigabyte respectively. (default: 4G) - **moveonenospc=BOOL|POLICY**: When enabled if a **write** fails with **ENOSPC** (no space left on device) or **EDQUOT** (disk quota exceeded) the policy selected will run to find a new location for the file. An attempt to move the file to that branch will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false, true = mfs) -- **inodecalc=passthrough|path-hash|devino-hash|hybrid-hash**: Selects - the inode calculation algorithm. (default: hybrid-hash) +- **[inodecalc](inodecalc.md)=passthrough|path-hash|devino-hash|hybrid-hash**: + Selects the inode calculation algorithm. (default: hybrid-hash) - **dropcacheonclose=BOOL**: When a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when @@ -43,12 +46,12 @@ These options are the same regardless of whether you use them with the shared mmap support. mergerfs will enable this feature if available but an option is provided to turn it off for testing and debugging purposes. (default: true) -- **symlinkify=BOOL**: When enabled and a file is not writable and its - mtime or ctime is older than **symlinkify_timeout** files will be - reported as symlinks to the original files. Please read more below - before using. (default: false) -- **symlinkify_timeout=UINT**: Time to wait, in seconds, to activate - the **symlinkify** behavior. (default: 3600) +- **[symlinkify](symlinkify.md)=BOOL**: When enabled and a file is not + writable and its mtime or ctime is older than **symlinkify_timeout** + files will be reported as symlinks to the original files. Please + read more below before using. (default: false) +- **[symlinkify_timeout](symlinkify.md)=UINT**: Time to wait, in + seconds, to activate the **symlinkify** behavior. (default: 3600) - **nullrw=BOOL**: Turns reads and writes into no-ops. The request will succeed but do nothing. Useful for benchmarking mergerfs. (default: false) @@ -62,23 +65,24 @@ These options are the same regardless of whether you use them with the link to always use the non-path preserving behavior. This means files, when renamed or linked, will stay on the same filesystem. (default: false) -- **export-support=BOOL**: Sets a low-level FUSE feature intended to - indicate the filesystem can support being exported via - NFS. (default: true) +- **[export-support](export-support.md)=BOOL**: Sets a low-level FUSE + feature intended to indicate the filesystem can support being + exported via NFS. (default: true) - **security_capability=BOOL**: If false return ENOATTR when xattr security.capability is queried. (default: true) -- **xattr=passthrough|noattr|nosys**: Runtime control of +- **[xattr](xattr.md)=passthrough|noattr|nosys**: Runtime control of xattrs. Default is to passthrough xattr requests. 'noattr' will short circuit as if nothing exists. 'nosys' will respond with ENOSYS as if xattrs are not supported or disabled. (default: passthrough) -- **link_cow=BOOL**: When enabled if a regular file is opened which - has a link count > 1 it will copy the file to a temporary file and - rename over the original. Breaking the link and providing a basic - copy-on-write function similar to cow-shell. (default: false) -- **statfs=base|full**: Controls how statfs works. 'base' means it - will always use all branches in statfs calculations. 'full' is in - effect path preserving and only includes branches where the path - exists. (default: base) +- **[link_cow](link_cow.md)=BOOL**: When enabled if a regular file is + opened which has a link count > 1 it will copy the file to a + temporary file and rename over the original. Breaking the link and + providing a basic copy-on-write function similar to + cow-shell. (default: false) +- **[statfs](statfs.md)=base|full**: Controls how statfs works. 'base' + means it will always use all branches in statfs calculations. 'full' + is in effect path preserving and only includes branches where the + path exists. (default: base) - **statfs_ignore=none|ro|nc**: 'ro' will cause statfs calculations to ignore available space for branches mounted or tagged as 'read-only' or 'no create'. 'nc' will ignore available space for branches tagged @@ -89,93 +93,99 @@ These options are the same regardless of whether you use them with the - **branches-mount-timeout=UINT**: Number of seconds to wait at startup for branches to be a mount other than the mountpoint's filesystem. (default: 0) -- **follow-symlinks=never|directory|regular|all**: Turns symlinks into - what they point to. (default: never) -- **link-exdev=passthrough|rel-symlink|abs-base-symlink|abs-pool-symlink**: +- **[follow-symlinks](follow-symlinks.md)=never|directory|regular|all**: + Turns symlinks into what they point to. (default: never) +- **[link-exdev](link-exdev.md)=passthrough|rel-symlink|abs-base-symlink|abs-pool-symlink**: When a link fails with EXDEV optionally create a symlink to the file instead. -- **rename-exdev=passthrough|rel-symlink|abs-symlink**: When a rename - fails with EXDEV optionally move the file to a special directory and - symlink to it. -- **readahead=UINT**: Set readahead (in kilobytes) for mergerfs and - branches if greater than 0. (default: 0) +- **[rename-exdev](rename-exdev.md)=passthrough|rel-symlink|abs-symlink**: + When a rename fails with EXDEV optionally move the file to a special + directory and symlink to it. +- **[readahead](readahead.md)=UINT**: Set readahead (in kilobytes) for + mergerfs and branches if greater than 0. (default: 0) - **posix_acl=BOOL**: Enable POSIX ACL support (if supported by kernel and underlying filesystem). (default: false) - **async_read=BOOL**: Perform reads asynchronously. If disabled or unavailable the kernel will ensure there is at most one pending read request per file handle and will attempt to order requests by offset. (default: true) -- **fuse_msg_size=UINT**: Set the max number of pages per FUSE - message. Only available on Linux >= 4.20 and ignored +- **[fuse_msg_size](fuse_msg_size.md)=UINT**: Set the max number of + pages per FUSE message. Only available on Linux >= 4.20 and ignored otherwise. (min: 1; max: 256; default: 256) -- **threads=INT**: Number of threads to use. When used alone - (`process-thread-count=-1`) it sets the number of threads reading - and processing FUSE messages. When used together it sets the number - of threads reading from FUSE. When set to zero it will attempt to - discover and use the number of logical cores. If the thread count is - set negative it will look up the number of cores then divide by the - absolute value. ie. threads=-2 on an 8 core machine will result in 8 - / 2 = 4 threads. There will always be at least 1 thread. If set to - -1 in combination with `process-thread-count` then it will try to - pick reasonable values based on CPU thread count. NOTE: higher - number of threads increases parallelism but usually decreases - throughput. (default: 0) -- **read-thread-count=INT**: Alias for `threads`. -- **process-thread-count=INT**: Enables separate thread pool to - asynchronously process FUSE requests. In this mode +- **[threads](threads.md)=INT**: Number of threads to use. When used + alone (`process-thread-count=-1`) it sets the number of threads + reading and processing FUSE messages. When used together it sets the + number of threads reading from FUSE. When set to zero it will + attempt to discover and use the number of logical cores. If the + thread count is set negative it will look up the number of cores + then divide by the absolute value. ie. threads=-2 on an 8 core + machine will result in 8 / 2 = 4 threads. There will always be at + least 1 thread. If set to -1 in combination with + `process-thread-count` then it will try to pick reasonable values + based on CPU thread count. NOTE: higher number of threads increases + parallelism but usually decreases throughput. (default: 0) +- **[read-thread-count](threads.md)=INT**: Alias for `threads`. +- **[process-thread-count](threads.md)=INT**: Enables separate thread + pool to asynchronously process FUSE requests. In this mode `read-thread-count` refers to the number of threads reading FUSE messages which are dispatched to process threads. -1 means disabled otherwise acts like `read-thread-count`. (default: -1) -- **process-thread-queue-depth=UINT**: Sets the number of requests any - single process thread can have queued up at one time. Meaning the - total memory usage of the queues is queue depth multiplied by the - number of process threads plus read thread count. 0 sets the depth - to the same as the process thread count. (default: 0) -- **pin-threads=STR**: Selects a strategy to pin threads to CPUs - (default: unset) -- **flush-on-close=never|always|opened-for-write**: Flush data cache - on file close. Mostly for when writeback is enabled or merging - network filesystems. (default: opened-for-write) +- **[process-thread-queue-depth](threads.md)=UINT**: Sets the number + of requests any single process thread can have queued up at one + time. Meaning the total memory usage of the queues is queue depth + multiplied by the number of process threads plus read thread + count. 0 sets the depth to the same as the process thread + count. (default: 0) +- **[pin-threads](pin-threads.md)=STR**: Selects a strategy to pin + threads to CPUs (default: unset) +- **[flush-on-close](flush-on-close.md)=never|always|opened-for-write**: + Flush data cache on file close. Mostly for when writeback is enabled + or merging network filesystems. (default: opened-for-write) - **scheduling-priority=INT**: Set mergerfs' scheduling priority. Valid values range from -20 to 19. See `setpriority` man page for more details. (default: -10) - **fsname=STR**: Sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed. -- **func.FUNC=POLICY**: Sets the specific FUSE function's policy. See - below for the list of value types. Example: **func.getattr=newest** -- **func.readdir=seq|cosr|cor|cosr:INT|cor:INT**: Sets `readdir` +- **[func.FUNC](functions_categories_policies.md)=POLICY**: Sets the + specific FUSE function's policy. See below for the list of value + types. Example: **func.getattr=newest** +- **[func.readdir](func_readdir.md)=seq|cosr|cor|cosr:INT|cor:INT**: Sets `readdir` policy. INT value sets the number of threads to use for concurrency. (default: seq) -- **category.action=POLICY**: Sets policy of all FUSE functions in the - action category. (default: epall) -- **category.create=POLICY**: Sets policy of all FUSE functions in the - create category. (default: epmfs) -- **category.search=POLICY**: Sets policy of all FUSE functions in the - search category. (default: ff) -- **cache.statfs=UINT**: 'statfs' cache timeout in seconds. (default: 0) -- **cache.attr=UINT**: File attribute cache timeout in - seconds. (default: 1) -- **cache.entry=UINT**: File name lookup cache timeout in - seconds. (default: 1) -- **cache.negative_entry=UINT**: Negative file name lookup cache +- **[category.action](functions_categories_policies.md)=POLICY**: Sets + policy of all FUSE functions in the action category. (default: + epall) +- **[category.create](functions_categories_policies.md)=POLICY**: Sets + policy of all FUSE functions in the create category. (default: + epmfs) +- **[category.search](functions_categories_policies.md)=POLICY**: Sets + policy of all FUSE functions in the search category. (default: ff) +- **[cache.statfs](cache.md#cachestatfs)=UINT**: 'statfs' cache timeout in seconds. (default: 0) -- **cache.files=libfuse|off|partial|full|auto-full|per-process**: File - page caching mode (default: libfuse) +- **[cache.attr](cache.md#cacheattr)=UINT**: File attribute cache + timeout in seconds. (default: 1) +- **[cache.entry](cache.md#cacheentry)=UINT**: File name lookup cache + timeout in seconds. (default: 1) +- **[cache.negative_entry](cache.md#cachenegative_entry)=UINT**: + Negative file name lookup cache timeout in seconds. (default: 0) +- **[cache.files](cache.md#cachefiles)=libfuse|off|partial|full|auto-full|per-process**: + File page caching mode (default: libfuse) - **cache.files.process-names=LIST**: A pipe | delimited list of process [comm](https://man7.org/linux/man-pages/man5/proc.5.html) names to enable page caching for when `cache.files=per-process`. (default: "rtorrent|qbittorrent-nox") -- **cache.writeback=BOOL**: Enable kernel writeback caching (default: - false) -- **cache.symlinks=BOOL**: Cache symlinks (if supported by kernel) - (default: false) -- **cache.readdir=BOOL**: Cache readdir (if supported by kernel) - (default: false) +- **[cache.writeback](cache.md#cachewriteback)=BOOL**: Enable kernel + writeback caching (default: false) +- **[cache.symlinks](cache.md#cachesymlinks)=BOOL**: Cache symlinks (if + supported by kernel) (default: false) +- **[cache.readdir](cache.md#cachereaddir)=BOOL**: Cache readdir (if + supported by kernel) (default: false) - **parallel-direct-writes=BOOL**: Allow the kernel to dispatch multiple, parallel (non-extending) write requests for files opened - with `cache.files=per-process` (if the process is not in `process-names`) - or `cache.files=off`. (This requires kernel support, and was added in v6.2) + with `cache.files=per-process` (if the process is not in + `process-names`) or `cache.files=off`. (Is a kernel feature added in + v6.2) (default: true) **NOTE:** Options are evaluated in the order listed so if the options are **func.rmdir=rand,category.action=ff** the **action** category diff --git a/mkdocs/docs/faq/why_isnt_it_working.md b/mkdocs/docs/faq/why_isnt_it_working.md index 8b9ec6e4..fb1afedb 100644 --- a/mkdocs/docs/faq/why_isnt_it_working.md +++ b/mkdocs/docs/faq/why_isnt_it_working.md @@ -34,7 +34,7 @@ the underlying branches. If you do not care about path preservation not](configuration_and_policies.md#how-can-i-ensure-files-are-collocated-on-the-same-branch)) and wish your files to be spread across all your filesystems change `create.category` to `pfrd`, `rand`, `mfs` or a similarly non-path -restrictive [policy](../config/functions_categories_and_policies.md). +restrictive [policy](../config/functions_categories_policies.md). ## Why isn't the create policy working? @@ -162,7 +162,7 @@ volumes: As described in the [rename and link](../config/rename_and_link.md) docs if a path-preserving create -[policy](../config/functions_categories_and_policies.md) is configured +[policy](../config/functions_categories_policies.md) is configured then mergerfs, in order to follow that restrictive policy, must return an error ([EXDEV](https://man7.org/linux/man-pages/man2/rename.2.html)) if the @@ -191,13 +191,13 @@ errors which `rename` and `link` may return even though they are perfectly valid situations and do not indicate actual device, filesystem, or OS problems. The error will rarely be returned by mergerfs if using a non-path preserving -[policy](../config/functions_categories_and_policies.md) however there -are edge cases where it may (such as mounting another filesystem into -the mergerfs pool.) If you do not care about path preservation set the +[policy](../config/functions_categories_policies.md) however there are +edge cases where it may (such as mounting another filesystem into the +mergerfs pool.) If you do not care about path preservation set the mergerfs policy to the non-path preserving variant. For example: `-o -category.create=pfrd`. Ideally the offending software would be fixed and -it is recommended that if you run into this problem you contact the -software's author and request proper handling of `EXDEV` errors. +category.create=pfrd`. Ideally the offending software would be fixed +and it is recommended that if you run into this problem you contact +the software's author and request proper handling of `EXDEV` errors. ### Additional Reading diff --git a/mkdocs/docs/index.md b/mkdocs/docs/index.md index 49611324..5d19d476 100644 --- a/mkdocs/docs/index.md +++ b/mkdocs/docs/index.md @@ -52,7 +52,7 @@ For functions which change attributes or remove the file the behavior may be applied to all instances found. Read more about [policies -here](config/functions_categories_and_policies.md). +here](config/functions_categories_policies.md). ### Visualization diff --git a/mkdocs/docs/intro_to_filesystems.md b/mkdocs/docs/intro_to_filesystems.md new file mode 100644 index 00000000..9bd71e1b --- /dev/null +++ b/mkdocs/docs/intro_to_filesystems.md @@ -0,0 +1,47 @@ +# Intro to Filesystems + +WORK IN PROGRESS + +mergerfs is a union filesystem that manages other filesystems. To +understand how mergerfs works and what the features do and mean you +must understand at least the basics of how filesystems work. Much of +the confusion with mergerfs is based in a lack of knowledge about +filesystems. This section of the documentation is provide a primer for +those needing that knowledge. + + +## terminology + + +## files + +### types + +* regular: +* directory: +* symlink: +* fifo: +* unix domain socket: + + +### inodes + +### names + +### permissions + +### owership + + +## functions + + +## workflows + +### creating a file + +### reading or writing to a file + +### copying a file + +### moving a file diff --git a/mkdocs/mkdocs.yml b/mkdocs/mkdocs.yml index 83e6c67c..f3f6d6d6 100644 --- a/mkdocs/mkdocs.yml +++ b/mkdocs/mkdocs.yml @@ -57,11 +57,12 @@ nav: - setup/upgrade.md - setup/build.md - terminology.md +- intro_to_filesystems.md - Config: - config/options.md - config/deprecated_options.md - config/branches.md - - config/functions_categories_and_policies.md + - config/functions_categories_policies.md - config/func_readdir.md - config/rename_and_link.md - config/cache.md