Browse Source

Merge pull request #46 from trapexit/action-search-merge

merge action and search category
pull/49/head
Antonio SJ Musumeci 10 years ago
parent
commit
dbe630c37d
  1. 28
      README.md
  2. 2
      src/category.cpp
  3. 4
      src/category.hpp
  4. 2
      src/chmod.cpp
  5. 2
      src/chown.cpp
  6. 2
      src/config.cpp
  7. 1
      src/config.hpp
  8. 4
      src/getxattr.cpp
  9. 2
      src/link.cpp
  10. 1
      src/listxattr.cpp
  11. 2
      src/option_parser.cpp
  12. 2
      src/removexattr.cpp
  13. 2
      src/rmdir.cpp
  14. 2
      src/setxattr.cpp
  15. 2
      src/truncate.cpp
  16. 2
      src/unlink.cpp
  17. 2
      src/utimens.cpp

28
README.md

@ -1,6 +1,6 @@
% mergerfs(1) mergerfs user manual % mergerfs(1) mergerfs user manual
% Antonio SJ Musumeci <trapexit@spawn.link> % Antonio SJ Musumeci <trapexit@spawn.link>
% June 9, 2014
% 2015-02-05
# NAME # NAME
@ -8,7 +8,7 @@ mergerfs - another FUSE union filesystem
# SYNOPSIS # SYNOPSIS
mergerfs -ocreate=epmfs,search=ff,action=ff &lt;srcpoints&gt; &lt;mountpoint&gt;
mergerfs -ocreate=epmfs,search=ff &lt;srcpoints&gt; &lt;mountpoint&gt;
# DESCRIPTION # DESCRIPTION
@ -23,7 +23,6 @@ Why create mergerfs when those exist? mhddfs isn't really maintained or flexible
| Option | Default | | Option | Default |
|--------|--------| |--------|--------|
| search | ff | | search | ff |
| action | ff |
| create | epmfs | | create | epmfs |
###srcpoints### ###srcpoints###
@ -47,13 +46,12 @@ In /etc/fstab it'd look like the following:
# POLICIES # POLICIES
Filesystem calls are broken up into 3 categories: search, action, and create. There are also some calls which have no policy attached due to state being kept between calls. These categories can be assigned a policy which dictates how [mergerfs](http://github.com/trapexit/mergerfs) behaves. Any policy can be assigned to a category though some aren't terribly practical. For instance: rand (Random) may be useful for **create** but could lead to very odd behavior if used for **search** or **action**.
Filesystem calls are broken up into 2 categories: search and create. There are also some calls which have no policy attached due to state being kept between calls. These categories can be assigned a policy which dictates how [mergerfs](http://github.com/trapexit/mergerfs) behaves. Any policy can be assigned to a category though some aren't terribly practical. For instance: rand (Random) may be useful for **create** but could lead to very odd behavior if used for **search**.
#### Functional classifications #### #### Functional classifications ####
| Class | FUSE calls | | Class | FUSE calls |
|-------|------------| |-------|------------|
| search | access, getattr, getxattr, listxattr, open, readlink |
| action | chmod, link, removexattr, rmdir, setxattr, truncate, unlink, utimens |
| search | access, getattr, getxattr, listxattr, open, readlink, chmod, link, removexattr, rmdir, setxattr, truncate, unlink, utimens |
| create | create, mkdir, mknod | | create | create, mkdir, mknod |
| none | fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, rename, statfs, symlink, write, release | | none | fallocate, fgetattr, fsync, ftruncate, ioctl, read, readdir, rename, statfs, symlink, write, release |
@ -67,11 +65,17 @@ Filesystem calls are broken up into 3 categories: search, action, and create. Th
| epmfs (existing path, most free space) | If the path exists in multiple locations use the one with the most free space. Otherwise fall back to mfs. | | epmfs (existing path, most free space) | If the path exists in multiple locations use the one with the most free space. Otherwise fall back to mfs. |
| rand (random) | Pick an existing destination at random. | | rand (random) | Pick an existing destination at random. |
#### readdir ####
[readdir](http://linux.die.net/man/3/readdir) is very different from most functions in this realm. It certainly could have it's own set of policies to tweak its behavior. At this time it provides a simple `first found` merging of directories and file found. That is: only the first file or directory found for a directory is returned.
It could be extended to offer the ability to see all files found. Perhaps concatinating `#` and a number to the name. But to really be useful you'd need to be able to access them which would complicate file lookup.
#### statvfs #### #### statvfs ####
It normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple points on the same drive will not result in double counting it's space.
[statvfs](http://linux.die.net/man/2/statvfs) normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple points on the same drive will not result in double counting it's space.
**NOTE:** create is really a search for existence and then create. The 'search' policy applies to the first part. If the [dirname](http://linux.die.net/man/3/dirname) of the full path is not found to exist [ENOENT](http://linux.die.net/man/3/errno) is returned.
**NOTE:** Since we can not (easily) replicate the atomicity of an `mkdir` or `mknod` without side effects those calls will first do a scan to see if the file exists and then attempts a create. This means there is a slight race condition. Worse case you'd end up with the directory or file on more than one mount.
# BUILDING # BUILDING
@ -98,22 +102,20 @@ Even if xattrs are disabled the [{list,get,set}xattrs](http://linux.die.net/man/
The keys are: The keys are:
* user.mergerfs.srcmounts * user.mergerfs.srcmounts
* user.mergerfs.action
* user.mergerfs.create * user.mergerfs.create
* user.mergerfs.search * user.mergerfs.search
``` ```
[trapexit:/tmp/mount] $ xattr -l .mergerfs [trapexit:/tmp/mount] $ xattr -l .mergerfs
user.mergerfs.srcmounts: /tmp/a:/tmp/b user.mergerfs.srcmounts: /tmp/a:/tmp/b
user.mergerfs.action: ff
user.mergerfs.create: epmfs user.mergerfs.create: epmfs
user.mergerfs.search: ff user.mergerfs.search: ff
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.action .mergerfs
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.search .mergerfs
ff ff
[trapexit:/tmp/mount] $ xattr -w user.mergerfs.action ffwp .mergerfs
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.action .mergerfs
[trapexit:/tmp/mount] $ xattr -w user.mergerfs.search ffwp .mergerfs
[trapexit:/tmp/mount] $ xattr -p user.mergerfs.search .mergerfs
ffwp ffwp
[trapexit:/tmp/mount] $ xattr -w user.mergerfs.srcmounts +/tmp/c .mergerfs [trapexit:/tmp/mount] $ xattr -w user.mergerfs.srcmounts +/tmp/c .mergerfs

2
src/category.cpp

@ -35,14 +35,12 @@ namespace mergerfs
const std::vector<Category> Category::_categories_ = const std::vector<Category> Category::_categories_ =
buildvector<Category,true> buildvector<Category,true>
(CATEGORY(invalid)) (CATEGORY(invalid))
(CATEGORY(action))
(CATEGORY(create)) (CATEGORY(create))
(CATEGORY(search)); (CATEGORY(search));
const Category * const Category::categories = &_categories_[1]; const Category * const Category::categories = &_categories_[1];
const Category &Category::invalid = Category::categories[Category::Enum::invalid]; const Category &Category::invalid = Category::categories[Category::Enum::invalid];
const Category &Category::action = Category::categories[Category::Enum::action];
const Category &Category::create = Category::categories[Category::Enum::create]; const Category &Category::create = Category::categories[Category::Enum::create];
const Category &Category::search = Category::categories[Category::Enum::search]; const Category &Category::search = Category::categories[Category::Enum::search];

4
src/category.hpp

@ -39,8 +39,7 @@ namespace mergerfs
{ {
invalid = -1, invalid = -1,
BEGIN = 0, BEGIN = 0,
action = BEGIN,
create,
create = BEGIN,
search, search,
END END
}; };
@ -89,7 +88,6 @@ namespace mergerfs
static const std::vector<Category> _categories_; static const std::vector<Category> _categories_;
static const Category * const categories; static const Category * const categories;
static const Category &invalid; static const Category &invalid;
static const Category &action;
static const Category &create; static const Category &create;
static const Category &search; static const Category &search;
}; };

2
src/chmod.cpp

@ -69,7 +69,7 @@ namespace mergerfs
const config::Config &config = config::get(); const config::Config &config = config::get();
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _chmod(*config.action,
return _chmod(*config.search,
config.srcmounts, config.srcmounts,
fusepath, fusepath,
mode); mode);

2
src/chown.cpp

@ -72,7 +72,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _chown(*config.action,
return _chown(*config.search,
config.srcmounts, config.srcmounts,
fusepath, fusepath,
uid, uid,

2
src/config.cpp

@ -43,14 +43,12 @@ namespace mergerfs
: destmount(), : destmount(),
srcmounts(), srcmounts(),
srcmountslock(), srcmountslock(),
action(policies[Category::Enum::action]),
create(policies[Category::Enum::create]), create(policies[Category::Enum::create]),
search(policies[Category::Enum::search]), search(policies[Category::Enum::search]),
controlfile("/.mergerfs") controlfile("/.mergerfs")
{ {
pthread_rwlock_init(&srcmountslock,NULL); pthread_rwlock_init(&srcmountslock,NULL);
action = &Policy::ff;
create = &Policy::epmfs; create = &Policy::epmfs;
search = &Policy::ff; search = &Policy::ff;
} }

1
src/config.hpp

@ -50,7 +50,6 @@ namespace mergerfs
mutable pthread_rwlock_t srcmountslock; mutable pthread_rwlock_t srcmountslock;
const Policy *policies[Category::Enum::END]; const Policy *policies[Category::Enum::END];
const Policy *&action;
const Policy *&create; const Policy *&create;
const Policy *&search; const Policy *&search;

4
src/getxattr.cpp

@ -52,9 +52,7 @@ _getxattr_controlfile(const Config &config,
size_t len; size_t len;
string attrvalue; string attrvalue;
if(attrname == "user.mergerfs.action")
attrvalue = (std::string)*config.action;
else if(attrname == "user.mergerfs.create")
if(attrname == "user.mergerfs.create")
attrvalue = (std::string)*config.create; attrvalue = (std::string)*config.create;
else if(attrname == "user.mergerfs.search") else if(attrname == "user.mergerfs.search")
attrvalue = (std::string)*config.search; attrvalue = (std::string)*config.search;

2
src/link.cpp

@ -91,7 +91,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _link(*config.action,
return _link(*config.search,
config.srcmounts, config.srcmounts,
from, from,
to); to);

1
src/listxattr.cpp

@ -49,7 +49,6 @@ _listxattr_controlfile(char *list,
{ {
const char xattrs[] = const char xattrs[] =
"user.mergerfs.srcmounts\0" "user.mergerfs.srcmounts\0"
"user.mergerfs.action\0"
"user.mergerfs.create\0" "user.mergerfs.create\0"
"user.mergerfs.search" "user.mergerfs.search"
; ;

2
src/option_parser.cpp

@ -57,8 +57,6 @@ process_opt(config::Config &config,
config.create = Policy::find(argvalue[1]); config.create = Policy::find(argvalue[1]);
else if(argvalue[0] == "search") else if(argvalue[0] == "search")
config.search = Policy::find(argvalue[1]); config.search = Policy::find(argvalue[1]);
else if(argvalue[0] == "action")
config.action = Policy::find(argvalue[1]);
else else
rv = 1; rv = 1;
break; break;

2
src/removexattr.cpp

@ -79,7 +79,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _removexattr(*config.action,
return _removexattr(*config.search,
config.srcmounts, config.srcmounts,
fusepath, fusepath,
attrname); attrname);

2
src/rmdir.cpp

@ -68,7 +68,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readguard(&config.srcmountslock); const rwlock::ReadGuard readguard(&config.srcmountslock);
return _rmdir(*config.action,
return _rmdir(*config.search,
config.srcmounts, config.srcmounts,
fusepath); fusepath);
} }

2
src/setxattr.cpp

@ -271,7 +271,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _setxattr(*config.action,
return _setxattr(*config.search,
config.srcmounts, config.srcmounts,
fusepath, fusepath,
attrname, attrname,

2
src/truncate.cpp

@ -71,7 +71,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _truncate(*config.action,
return _truncate(*config.search,
config.srcmounts, config.srcmounts,
fusepath, fusepath,
size); size);

2
src/unlink.cpp

@ -68,7 +68,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _unlink(*config.action,
return _unlink(*config.search,
config.srcmounts, config.srcmounts,
fusepath); fusepath);
} }

2
src/utimens.cpp

@ -71,7 +71,7 @@ namespace mergerfs
const ugid::SetResetGuard ugid(fc->uid,fc->gid); const ugid::SetResetGuard ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.srcmountslock); const rwlock::ReadGuard readlock(&config.srcmountslock);
return _utimens(*config.action,
return _utimens(*config.search,
config.srcmounts, config.srcmounts,
fusepath, fusepath,
ts); ts);

Loading…
Cancel
Save