Browse Source

Merge pull request #565 from trapexit/open_cache

add policy cache for 'open'
pull/567/head
trapexit 6 years ago
committed by GitHub
parent
commit
f42bd8666e
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
  1. 10
      README.md
  2. 2
      libfuse/configure.ac
  3. 20
      man/mergerfs.1
  4. 2
      src/access.cpp
  5. 69
      src/category.cpp
  6. 105
      src/category.hpp
  7. 1
      src/chmod.cpp
  8. 1
      src/chown.cpp
  9. 4
      src/config.hpp
  10. 1
      src/getattr.cpp
  11. 2
      src/getxattr.cpp
  12. 1
      src/link.cpp
  13. 22
      src/listxattr.cpp
  14. 12
      src/open.cpp
  15. 16
      src/option_parser.cpp
  16. 102
      src/policy.cpp
  17. 322
      src/policy.hpp
  18. 21
      src/policy_all.cpp
  19. 128
      src/policy_cache.cpp
  20. 61
      src/policy_cache.hpp
  21. 35
      src/policy_epall.cpp
  22. 36
      src/policy_epff.cpp
  23. 36
      src/policy_eplfs.cpp
  24. 36
      src/policy_eplus.cpp
  25. 36
      src/policy_epmfs.cpp
  26. 29
      src/policy_eprand.cpp
  27. 17
      src/policy_erofs.cpp
  28. 21
      src/policy_ff.cpp
  29. 17
      src/policy_invalid.cpp
  30. 22
      src/policy_lfs.cpp
  31. 22
      src/policy_lus.cpp
  32. 22
      src/policy_mfs.cpp
  33. 35
      src/policy_newest.cpp
  34. 29
      src/policy_rand.cpp
  35. 1
      src/readlink.cpp
  36. 49
      src/release.cpp
  37. 29
      src/releasedir.cpp
  38. 1
      src/removexattr.cpp
  39. 2
      src/rename.cpp
  40. 1
      src/rmdir.cpp
  41. 11
      src/setxattr.cpp
  42. 1
      src/truncate.cpp
  43. 106
      src/unlink.cpp
  44. 1
      src/utimens.cpp

10
README.md

@ -84,6 +84,7 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs**
* **fsname=name**: sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed. * **fsname=name**: sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed.
* **func.<func>=<policy>**: sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest** * **func.<func>=<policy>**: sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest**
* **category.<category>=<policy>**: Sets policy of all FUSE functions in the provided category. Example: **category.create=mfs** * **category.<category>=<policy>**: Sets policy of all FUSE functions in the provided category. Example: **category.create=mfs**
* **cache.open=<int>**: 'open' policy cache timeout in seconds. (default: 0)
**NOTE:** Options are evaluated in the order listed so if the options are **func.rmdir=rand,category.action=ff** the **action** category setting will override the **rmdir** setting. **NOTE:** Options are evaluated in the order listed so if the options are **func.rmdir=rand,category.action=ff** the **action** category setting will override the **rmdir** setting.
@ -491,6 +492,15 @@ It's a difficult balance between memory usage, cache bloat & duplication, and pe
Given the relatively high cost of FUSE due to the kernel <-> userspace round trips there are kernel side caches for file entries and attributes. The entry cache limits the `lookup` calls to mergerfs which ask if a file exists. The attribute cache limits the need to make `getattr` calls to mergerfs which provide file attributes (mode, size, type, etc.). As with the page cache these should not be used if the underlying filesystems are being manipulated at the same time as it could lead to odd behavior or data corruption. The options for setting these are `entry_timeout` and `negative_timeout` for the entry cache and `attr_timeout` for the attributes cache. `negative_timeout` refers to the timeout for negative responses to lookups (non-existant files). Given the relatively high cost of FUSE due to the kernel <-> userspace round trips there are kernel side caches for file entries and attributes. The entry cache limits the `lookup` calls to mergerfs which ask if a file exists. The attribute cache limits the need to make `getattr` calls to mergerfs which provide file attributes (mode, size, type, etc.). As with the page cache these should not be used if the underlying filesystems are being manipulated at the same time as it could lead to odd behavior or data corruption. The options for setting these are `entry_timeout` and `negative_timeout` for the entry cache and `attr_timeout` for the attributes cache. `negative_timeout` refers to the timeout for negative responses to lookups (non-existant files).
#### policy caching
Policies are run every time a function is called. These policies can be expensive depending on the setup and usage patterns. Generally we wouldn't want to cache policy results because it may result in stale responses if the underlying drives are used directly.
The `open` policy cache will cache the result of an `open` policy for a particular input for `cache.open` seconds or until the file is unlinked. Each file close (release) will randomly chose to clean up the cache of expired entries.
This cache is useful in cases like that of **Transmission** which has a "open, read/write, close" pattern (which is much more costly due to the FUSE overhead than normal.)
#### writeback caching #### writeback caching
writeback caching is a technique for improving write speeds by batching writes at a faster device and then bulk writing to the slower device. With FUSE the kernel will wait for a number of writes to be made and then send it to the filesystem as one request. mergerfs currently uses a slightly modified and vendored libfuse 2.9.7 which does not support writeback caching. However, a prototype port to libfuse 3.x has been made and the writeback cache appears to work as expected (though performance improvements greatly depend on the way the client app writes data). Once the port is complete and thoroughly tested writeback caching will be available. writeback caching is a technique for improving write speeds by batching writes at a faster device and then bulk writing to the slower device. With FUSE the kernel will wait for a number of writes to be made and then send it to the filesystem as one request. mergerfs currently uses a slightly modified and vendored libfuse 2.9.7 which does not support writeback caching. However, a prototype port to libfuse 3.x has been made and the writeback cache appears to work as expected (though performance improvements greatly depend on the way the client app writes data). Once the port is complete and thoroughly tested writeback caching will be available.

2
libfuse/configure.ac

@ -1,4 +1,4 @@
AC_INIT(fuse, 2.9.8-mergerfs)
AC_INIT(fuse, 2.9.7-mergerfs_2.26.0)
AC_PREREQ(2.59d) AC_PREREQ(2.59d)
AC_CONFIG_MACRO_DIR([m4]) AC_CONFIG_MACRO_DIR([m4])

20
man/mergerfs.1

@ -211,6 +211,9 @@ Example: \f[B]func.getattr=newest\f[]
\f[B]category.<category>=<policy>\f[]: Sets policy of all FUSE functions \f[B]category.<category>=<policy>\f[]: Sets policy of all FUSE functions
in the provided category. in the provided category.
Example: \f[B]category.create=mfs\f[] Example: \f[B]category.create=mfs\f[]
.IP \[bu] 2
\f[B]cache.open=\f[]: \[aq]open\[aq] policy cache timeout in seconds.
(default: 0)
.PP .PP
\f[B]NOTE:\f[] Options are evaluated in the order listed so if the \f[B]NOTE:\f[] Options are evaluated in the order listed so if the
options are \f[B]func.rmdir=rand,category.action=ff\f[] the options are \f[B]func.rmdir=rand,category.action=ff\f[] the
@ -1034,6 +1037,23 @@ The options for setting these are \f[C]entry_timeout\f[] and
for the attributes cache. for the attributes cache.
\f[C]negative_timeout\f[] refers to the timeout for negative responses \f[C]negative_timeout\f[] refers to the timeout for negative responses
to lookups (non\-existant files). to lookups (non\-existant files).
.SS policy caching
.PP
Policies are run every time a function is called.
These policies can be expensive depending on the setup and usage
patterns.
Generally we wouldn\[aq]t want to cache policy results because it may
result in stale responses if the underlying drives are used directly.
.PP
The \f[C]open\f[] policy cache will cache the result of an \f[C]open\f[]
policy for a particular input for \f[C]cache.open\f[] seconds or until
the file is unlinked.
Each file close (release) will randomly chose to clean up the cache of
expired entries.
.PP
This cache is useful in cases like that of \f[B]Transmission\f[] which
has a "open, read/write, close" pattern (which is much more costly due
to the FUSE overhead than normal.)
.SS writeback caching .SS writeback caching
.PP .PP
writeback caching is a technique for improving write speeds by batching writeback caching is a technique for improving write speeds by batching

2
src/access.cpp

@ -26,8 +26,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
using mergerfs::Category;
static static
int int

69
src/category.cpp

@ -22,41 +22,38 @@
#define CATEGORY(X) Category(Category::Enum::X,#X) #define CATEGORY(X) Category(Category::Enum::X,#X)
namespace mergerfs
const std::vector<Category> Category::_categories_ =
buildvector<Category,true>
(CATEGORY(invalid))
(CATEGORY(action))
(CATEGORY(create))
(CATEGORY(search));
const Category * const Category::categories = &_categories_[1];
const Category &Category::invalid = Category::categories[Category::Enum::invalid];
const Category &Category::action = Category::categories[Category::Enum::action];
const Category &Category::create = Category::categories[Category::Enum::create];
const Category &Category::search = Category::categories[Category::Enum::search];
const Category&
Category::find(const std::string &str)
{ {
const std::vector<Category> Category::_categories_ =
buildvector<Category,true>
(CATEGORY(invalid))
(CATEGORY(action))
(CATEGORY(create))
(CATEGORY(search));
const Category * const Category::categories = &_categories_[1];
const Category &Category::invalid = Category::categories[Category::Enum::invalid];
const Category &Category::action = Category::categories[Category::Enum::action];
const Category &Category::create = Category::categories[Category::Enum::create];
const Category &Category::search = Category::categories[Category::Enum::search];
const Category&
Category::find(const std::string &str)
{
for(int i = Enum::BEGIN; i != Enum::END; ++i)
{
if(categories[i] == str)
return categories[i];
}
return invalid;
}
const Category&
Category::find(const Category::Enum::Type i)
{
if(i >= Category::Enum::BEGIN &&
i < Category::Enum::END)
return categories[i];
return invalid;
}
for(int i = Enum::BEGIN; i != Enum::END; ++i)
{
if(categories[i] == str)
return categories[i];
}
return invalid;
}
const Category&
Category::find(const Category::Enum::Type i)
{
if(i >= Category::Enum::BEGIN &&
i < Category::Enum::END)
return categories[i];
return invalid;
} }

105
src/category.hpp

@ -19,69 +19,66 @@
#include <string> #include <string>
#include <vector> #include <vector>
namespace mergerfs
class Category
{ {
class Category
public:
struct Enum
{ {
public:
struct Enum
{
enum Type
{
invalid = -1,
BEGIN = 0,
action = BEGIN,
create,
search,
END
};
};
enum Type
{
invalid = -1,
BEGIN = 0,
action = BEGIN,
create,
search,
END
};
};
private:
Enum::Type _enum;
std::string _str;
private:
Enum::Type _enum;
std::string _str;
public:
Category()
: _enum(invalid),
_str(invalid)
{
}
public:
Category()
: _enum(invalid),
_str(invalid)
{
}
Category(const Enum::Type enum_,
const std::string &str_)
: _enum(enum_),
_str(str_)
{
}
Category(const Enum::Type enum_,
const std::string &str_)
: _enum(enum_),
_str(str_)
{
}
public:
operator const Enum::Type() const { return _enum; }
operator const std::string&() const { return _str; }
operator const Category*() const { return this; }
public:
operator const Enum::Type() const { return _enum; }
operator const std::string&() const { return _str; }
operator const Category*() const { return this; }
bool operator==(const std::string &str_) const
{ return _str == str_; }
bool operator==(const std::string &str_) const
{ return _str == str_; }
bool operator==(const Enum::Type enum_) const
{ return _enum == enum_; }
bool operator==(const Enum::Type enum_) const
{ return _enum == enum_; }
bool operator!=(const Category &r) const
{ return _enum != r._enum; }
bool operator!=(const Category &r) const
{ return _enum != r._enum; }
bool operator<(const Category &r) const
{ return _enum < r._enum; }
bool operator<(const Category &r) const
{ return _enum < r._enum; }
public:
static const Category &find(const std::string&);
static const Category &find(const Enum::Type);
public:
static const Category &find(const std::string&);
static const Category &find(const Enum::Type);
public:
static const std::vector<Category> _categories_;
static const Category * const categories;
static const Category &invalid;
static const Category &action;
static const Category &create;
static const Category &search;
};
}
public:
static const std::vector<Category> _categories_;
static const Category * const categories;
static const Category &invalid;
static const Category &action;
static const Category &create;
static const Category &search;
};

1
src/chmod.cpp

@ -29,7 +29,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

1
src/chown.cpp

@ -29,7 +29,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
using mergerfs::Config; using mergerfs::Config;
static static

4
src/config.hpp

@ -19,6 +19,7 @@
#include "branch.hpp" #include "branch.hpp"
#include "fusefunc.hpp" #include "fusefunc.hpp"
#include "policy.hpp" #include "policy.hpp"
#include "policy_cache.hpp"
#include <fuse.h> #include <fuse.h>
@ -102,6 +103,9 @@ namespace mergerfs
const Policy *&unlink; const Policy *&unlink;
const Policy *&utimens; const Policy *&utimens;
public:
mutable PolicyCache open_cache;
public: public:
const std::string controlfile; const std::string controlfile;

1
src/getattr.cpp

@ -30,7 +30,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

2
src/getxattr.cpp

@ -306,6 +306,8 @@ _getxattr_controlfile(const Config &config,
_getxattr_controlfile_category_policy(config,attr[3],attrvalue); _getxattr_controlfile_category_policy(config,attr[3],attrvalue);
else if(attr[2] == "func") else if(attr[2] == "func")
_getxattr_controlfile_fusefunc_policy(config,attr[3],attrvalue); _getxattr_controlfile_fusefunc_policy(config,attr[3],attrvalue);
else if((attr[2] == "cache") && (attr[3] == "open"))
_getxattr_controlfile_uint64_t(config.open_cache.timeout,attrvalue);
break; break;
} }

1
src/link.cpp

@ -30,7 +30,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
using namespace mergerfs; using namespace mergerfs;
static static

22
src/listxattr.cpp

@ -43,24 +43,26 @@ _listxattr_controlfile(char *list,
string xattrs; string xattrs;
const vector<string> strs = const vector<string> strs =
buildvector<string> buildvector<string>
("user.mergerfs.srcmounts")
("user.mergerfs.branches") ("user.mergerfs.branches")
("user.mergerfs.minfreespace")
("user.mergerfs.moveonenospc")
("user.mergerfs.cache.open")
("user.mergerfs.direct_io")
("user.mergerfs.dropcacheonclose") ("user.mergerfs.dropcacheonclose")
("user.mergerfs.symlinkify")
("user.mergerfs.symlinkify_timeout")
("user.mergerfs.nullrw")
("user.mergerfs.ignorepponrename") ("user.mergerfs.ignorepponrename")
("user.mergerfs.link_cow") ("user.mergerfs.link_cow")
("user.mergerfs.minfreespace")
("user.mergerfs.moveonenospc")
("user.mergerfs.nullrw")
("user.mergerfs.pid")
("user.mergerfs.policies")
("user.mergerfs.security_capability") ("user.mergerfs.security_capability")
("user.mergerfs.xattr")
("user.mergerfs.srcmounts")
("user.mergerfs.statfs") ("user.mergerfs.statfs")
("user.mergerfs.statfs_ignore") ("user.mergerfs.statfs_ignore")
("user.mergerfs.direct_io")
("user.mergerfs.policies")
("user.mergerfs.symlinkify")
("user.mergerfs.symlinkify_timeout")
("user.mergerfs.version") ("user.mergerfs.version")
("user.mergerfs.pid");
("user.mergerfs.xattr")
;
xattrs.reserve(1024); xattrs.reserve(1024);
for(size_t i = 0; i < strs.size(); i++) for(size_t i = 0; i < strs.size(); i++)

12
src/open.cpp

@ -20,6 +20,7 @@
#include "fs_base_open.hpp" #include "fs_base_open.hpp"
#include "fs_cow.hpp" #include "fs_cow.hpp"
#include "fs_path.hpp" #include "fs_path.hpp"
#include "policy_cache.hpp"
#include "rwlock.hpp" #include "rwlock.hpp"
#include "ugid.hpp" #include "ugid.hpp"
@ -28,11 +29,8 @@
#include <string> #include <string>
#include <vector> #include <vector>
#include <fcntl.h>
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
namespace local namespace local
{ {
@ -64,6 +62,7 @@ namespace local
static static
int int
open(Policy::Func::Search searchFunc_, open(Policy::Func::Search searchFunc_,
PolicyCache &cache,
const Branches &branches_, const Branches &branches_,
const uint64_t minfreespace_, const uint64_t minfreespace_,
const char *fusepath_, const char *fusepath_,
@ -72,13 +71,13 @@ namespace local
uint64_t *fh_) uint64_t *fh_)
{ {
int rv; int rv;
vector<const string*> basepaths;
string basepath;
rv = searchFunc_(branches_,fusepath_,minfreespace_,basepaths);
rv = cache(searchFunc_,branches_,fusepath_,minfreespace_,&basepath);
if(rv == -1) if(rv == -1)
return -errno; return -errno;
return local::open_core(*basepaths[0],fusepath_,flags_,link_cow_,fh_);
return local::open_core(basepath,fusepath_,flags_,link_cow_,fh_);
} }
} }
@ -97,6 +96,7 @@ namespace mergerfs
ffi_->direct_io = config.direct_io; ffi_->direct_io = config.direct_io;
return local::open(config.open, return local::open(config.open,
config.open_cache,
config.branches, config.branches,
config.minfreespace, config.minfreespace,
fusepath_, fusepath_,

16
src/option_parser.cpp

@ -202,6 +202,18 @@ parse_and_process_statfsignore(const std::string &value_,
return 0; return 0;
} }
static
int
parse_and_process_cache(Config &config_,
const string &func_,
const string &value_)
{
if(func_ == "open")
return parse_and_process(value_,config_.open_cache.timeout);
return 1;
}
static static
int int
parse_and_process_arg(Config &config, parse_and_process_arg(Config &config,
@ -233,6 +245,8 @@ parse_and_process_kv_arg(Config &config,
rv = config.set_func_policy(keypart[1],value); rv = config.set_func_policy(keypart[1],value);
else if(keypart[0] == "category") else if(keypart[0] == "category")
rv = config.set_category_policy(keypart[1],value); rv = config.set_category_policy(keypart[1],value);
else if(keypart[0] == "cache")
rv = parse_and_process_cache(config,keypart[1],value);
} }
else else
{ {
@ -336,6 +350,8 @@ usage(void)
" splice_write, splice_move\n" " splice_write, splice_move\n"
" -o func.<f>=<p> Set function <f> to policy <p>\n" " -o func.<f>=<p> Set function <f> to policy <p>\n"
" -o category.<c>=<p> Set functions in category <c> to <p>\n" " -o category.<c>=<p> Set functions in category <c> to <p>\n"
" -o cache.open=<int> 'open' policy cache timeout in seconds.\n"
" default = 0 (disabled)\n"
" -o direct_io Bypass page caching, may increase write\n" " -o direct_io Bypass page caching, may increase write\n"
" speeds at the cost of reads. Please read docs\n" " speeds at the cost of reads. Please read docs\n"
" for more details as there are tradeoffs.\n" " for more details as there are tradeoffs.\n"

102
src/policy.cpp

@ -24,65 +24,63 @@
#define PRESERVES_PATH true #define PRESERVES_PATH true
#define DOESNT_PRESERVE_PATH false #define DOESNT_PRESERVE_PATH false
namespace mergerfs
{
const std::vector<Policy> Policy::_policies_ =
buildvector<Policy,true>
(POLICY(invalid,DOESNT_PRESERVE_PATH))
(POLICY(all,DOESNT_PRESERVE_PATH))
(POLICY(epall,PRESERVES_PATH))
(POLICY(epff,PRESERVES_PATH))
(POLICY(eplfs,PRESERVES_PATH))
(POLICY(eplus,PRESERVES_PATH))
(POLICY(epmfs,PRESERVES_PATH))
(POLICY(eprand,PRESERVES_PATH))
(POLICY(erofs,DOESNT_PRESERVE_PATH))
(POLICY(ff,DOESNT_PRESERVE_PATH))
(POLICY(lfs,DOESNT_PRESERVE_PATH))
(POLICY(lus,DOESNT_PRESERVE_PATH))
(POLICY(mfs,DOESNT_PRESERVE_PATH))
(POLICY(newest,DOESNT_PRESERVE_PATH))
(POLICY(rand,DOESNT_PRESERVE_PATH));
const std::vector<Policy> Policy::_policies_ =
buildvector<Policy,true>
(POLICY(invalid,DOESNT_PRESERVE_PATH))
(POLICY(all,DOESNT_PRESERVE_PATH))
(POLICY(epall,PRESERVES_PATH))
(POLICY(epff,PRESERVES_PATH))
(POLICY(eplfs,PRESERVES_PATH))
(POLICY(eplus,PRESERVES_PATH))
(POLICY(epmfs,PRESERVES_PATH))
(POLICY(eprand,PRESERVES_PATH))
(POLICY(erofs,DOESNT_PRESERVE_PATH))
(POLICY(ff,DOESNT_PRESERVE_PATH))
(POLICY(lfs,DOESNT_PRESERVE_PATH))
(POLICY(lus,DOESNT_PRESERVE_PATH))
(POLICY(mfs,DOESNT_PRESERVE_PATH))
(POLICY(newest,DOESNT_PRESERVE_PATH))
(POLICY(rand,DOESNT_PRESERVE_PATH));
const Policy * const Policy::policies = &_policies_[1];
const Policy * const Policy::policies = &_policies_[1];
#define CONST_POLICY(X) const Policy &Policy::X = Policy::policies[Policy::Enum::X] #define CONST_POLICY(X) const Policy &Policy::X = Policy::policies[Policy::Enum::X]
CONST_POLICY(invalid);
CONST_POLICY(all);
CONST_POLICY(epall);
CONST_POLICY(epff);
CONST_POLICY(eplfs);
CONST_POLICY(eplus);
CONST_POLICY(epmfs);
CONST_POLICY(eprand);
CONST_POLICY(erofs);
CONST_POLICY(ff);
CONST_POLICY(lfs);
CONST_POLICY(lus);
CONST_POLICY(mfs);
CONST_POLICY(newest);
CONST_POLICY(rand);
CONST_POLICY(invalid);
CONST_POLICY(all);
CONST_POLICY(epall);
CONST_POLICY(epff);
CONST_POLICY(eplfs);
CONST_POLICY(eplus);
CONST_POLICY(epmfs);
CONST_POLICY(eprand);
CONST_POLICY(erofs);
CONST_POLICY(ff);
CONST_POLICY(lfs);
CONST_POLICY(lus);
CONST_POLICY(mfs);
CONST_POLICY(newest);
CONST_POLICY(rand);
const Policy&
Policy::find(const std::string &str)
const Policy&
Policy::find(const std::string &str)
{
for(int i = Enum::BEGIN; i != Enum::END; ++i)
{ {
for(int i = Enum::BEGIN; i != Enum::END; ++i)
{
if(policies[i] == str)
return policies[i];
}
return invalid;
if(policies[i] == str)
return policies[i];
} }
const Policy&
Policy::find(const Policy::Enum::Type i)
{
if(i >= Policy::Enum::BEGIN &&
i < Policy::Enum::END)
return policies[i];
return invalid;
}
return invalid;
}
const Policy&
Policy::find(const Policy::Enum::Type i)
{
if(i >= Policy::Enum::BEGIN &&
i < Policy::Enum::END)
return policies[i];
return invalid;
} }

322
src/policy.hpp

@ -24,168 +24,178 @@
#include <string> #include <string>
#include <vector> #include <vector>
namespace mergerfs
class Policy
{ {
class Policy
public:
struct Enum
{ {
public:
struct Enum
{
enum Type
{
invalid = -1,
BEGIN = 0,
all = BEGIN,
epall,
epff,
eplfs,
eplus,
epmfs,
eprand,
erofs,
ff,
lfs,
lus,
mfs,
newest,
rand,
END
};
static size_t begin() { return BEGIN; }
static size_t end() { return END; }
};
struct Func
{
typedef std::string string;
typedef std::vector<string> strvec;
typedef std::vector<const string*> cstrptrvec;
typedef const string cstring;
typedef const uint64_t cuint64_t;
typedef const strvec cstrvec;
typedef const Category::Enum::Type CType;
typedef int (*Ptr)(CType,const Branches &,const char *,cuint64_t,cstrptrvec &);
template <CType T>
class Base
enum Type
{ {
public:
Base(const Policy *p)
: func(p->_func)
{}
int
operator()(const Branches &b,const char *c,cuint64_t d,cstrptrvec &e)
{
return func(T,b,c,d,e);
}
int
operator()(const Branches &b,const string &c,cuint64_t d,cstrptrvec &e)
{
return func(T,b,c.c_str(),d,e);
}
private:
const Ptr func;
invalid = -1,
BEGIN = 0,
all = BEGIN,
epall,
epff,
eplfs,
eplus,
epmfs,
eprand,
erofs,
ff,
lfs,
lus,
mfs,
newest,
rand,
END
}; };
typedef Base<Category::Enum::action> Action;
typedef Base<Category::Enum::create> Create;
typedef Base<Category::Enum::search> Search;
static int invalid(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int all(CType,const Branches&,const char*,cuint64_t,cstrptrvec&);
static int epall(CType,const Branches&,const char*,cuint64_t,cstrptrvec&);
static int epff(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int eplfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int eplus(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int epmfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int eprand(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int erofs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int ff(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int lfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int lus(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int mfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int newest(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int rand(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
};
static size_t begin() { return BEGIN; }
static size_t end() { return END; }
};
private:
Enum::Type _enum;
std::string _str;
Func::Ptr _func;
bool _path_preserving;
public:
Policy()
: _enum(invalid),
_str(invalid),
_func(invalid),
_path_preserving(false)
{
}
Policy(const Enum::Type enum_,
const std::string &str_,
const Func::Ptr func_,
const bool path_preserving_)
: _enum(enum_),
_str(str_),
_func(func_),
_path_preserving(path_preserving_)
struct Func
{
typedef std::string string;
typedef std::vector<string> strvec;
typedef std::vector<const string*> cstrptrvec;
typedef const string cstring;
typedef const uint64_t cuint64_t;
typedef const strvec cstrvec;
typedef const Category::Enum::Type CType;
typedef int (*Ptr)(CType,const Branches &,const char *,cuint64_t,cstrptrvec &);
template <CType T>
class Base
{ {
}
public:
Base(const Policy *p)
: func(p->_func)
{}
bool
path_preserving() const
{
return _path_preserving;
}
public:
operator const Enum::Type() const { return _enum; }
operator const std::string&() const { return _str; }
operator const Func::Ptr() const { return _func; }
operator const Policy*() const { return this; }
bool operator==(const Enum::Type enum_) const
{ return _enum == enum_; }
bool operator==(const std::string &str_) const
{ return _str == str_; }
bool operator==(const Func::Ptr func_) const
{ return _func == func_; }
bool operator!=(const Policy &r) const
{ return _enum != r._enum; }
bool operator<(const Policy &r) const
{ return _enum < r._enum; }
public:
static const Policy &find(const std::string&);
static const Policy &find(const Enum::Type);
public:
static const std::vector<Policy> _policies_;
static const Policy * const policies;
static const Policy &invalid;
static const Policy &all;
static const Policy &epall;
static const Policy &epff;
static const Policy &eplfs;
static const Policy &eplus;
static const Policy &epmfs;
static const Policy &eprand;
static const Policy &erofs;
static const Policy &ff;
static const Policy &lfs;
static const Policy &lus;
static const Policy &mfs;
static const Policy &newest;
static const Policy &rand;
int
operator()(const Branches &b,const char *c,cuint64_t d,cstrptrvec &e)
{
return func(T,b,c,d,e);
}
int
operator()(const Branches &b,const string &c,cuint64_t d,cstrptrvec &e)
{
return func(T,b,c.c_str(),d,e);
}
int
operator()(const Branches &b,const char *c,cuint64_t d,string *e)
{
int rv;
cstrptrvec v;
rv = func(T,b,c,d,v);
if(!v.empty())
*e = *v[0];
return rv;
}
private:
const Ptr func;
};
typedef Base<Category::Enum::action> Action;
typedef Base<Category::Enum::create> Create;
typedef Base<Category::Enum::search> Search;
static int invalid(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int all(CType,const Branches&,const char*,cuint64_t,cstrptrvec&);
static int epall(CType,const Branches&,const char*,cuint64_t,cstrptrvec&);
static int epff(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int eplfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int eplus(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int epmfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int eprand(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int erofs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int ff(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int lfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int lus(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int mfs(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int newest(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
static int rand(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
}; };
}
private:
Enum::Type _enum;
std::string _str;
Func::Ptr _func;
bool _path_preserving;
public:
Policy()
: _enum(invalid),
_str(invalid),
_func(invalid),
_path_preserving(false)
{
}
Policy(const Enum::Type enum_,
const std::string &str_,
const Func::Ptr func_,
const bool path_preserving_)
: _enum(enum_),
_str(str_),
_func(func_),
_path_preserving(path_preserving_)
{
}
bool
path_preserving() const
{
return _path_preserving;
}
public:
operator const Enum::Type() const { return _enum; }
operator const std::string&() const { return _str; }
operator const Func::Ptr() const { return _func; }
operator const Policy*() const { return this; }
bool operator==(const Enum::Type enum_) const
{ return _enum == enum_; }
bool operator==(const std::string &str_) const
{ return _str == str_; }
bool operator==(const Func::Ptr func_) const
{ return _func == func_; }
bool operator!=(const Policy &r) const
{ return _enum != r._enum; }
bool operator<(const Policy &r) const
{ return _enum < r._enum; }
public:
static const Policy &find(const std::string&);
static const Policy &find(const Enum::Type);
public:
static const std::vector<Policy> _policies_;
static const Policy * const policies;
static const Policy &invalid;
static const Policy &all;
static const Policy &epall;
static const Policy &epff;
static const Policy &eplfs;
static const Policy &eplus;
static const Policy &epmfs;
static const Policy &eprand;
static const Policy &erofs;
static const Policy &ff;
static const Policy &lfs;
static const Policy &lus;
static const Policy &mfs;
static const Policy &newest;
static const Policy &rand;
};

21
src/policy_all.cpp

@ -65,18 +65,15 @@ namespace all
} }
} }
namespace mergerfs
int
Policy::Func::all(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::all(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
if(type == Category::Enum::create)
return all::create(branches_,minfreespace,paths);
if(type == Category::Enum::create)
return all::create(branches_,minfreespace,paths);
return Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
}
return Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
} }

128
src/policy_cache.cpp

@ -0,0 +1,128 @@
#include "policy_cache.hpp"
#include <cstdlib>
#include <map>
#include <string>
#include <vector>
#include <sys/time.h>
using std::map;
using std::string;
using std::vector;
static const uint64_t DEFAULT_TIMEOUT = 0;
namespace local
{
static
uint64_t
get_time(void)
{
uint64_t rv;
struct timeval now;
::gettimeofday(&now,NULL);
rv = now.tv_sec;
return rv;
}
}
PolicyCache::Value::Value()
: time(0),
path()
{
}
PolicyCache::PolicyCache(void)
: timeout(DEFAULT_TIMEOUT)
{
}
void
PolicyCache::erase(const char *fusepath_)
{
pthread_mutex_lock(&_lock);
_cache.erase(fusepath_);
pthread_mutex_unlock(&_lock);
}
void
PolicyCache::cleanup(const int prob_)
{
uint64_t now;
map<string,Value>::iterator i;
if(rand() % prob_)
return;
now = local::get_time();
pthread_mutex_lock(&_lock);
i = _cache.begin();
while(i != _cache.end())
{
if((now - i->second.time) >= timeout)
_cache.erase(i++);
else
++i;
}
pthread_mutex_unlock(&_lock);
}
void
PolicyCache::clear(void)
{
pthread_mutex_lock(&_lock);
_cache.clear();
pthread_mutex_unlock(&_lock);
}
int
PolicyCache::operator()(Policy::Func::Search &func_,
const Branches &branches_,
const char *fusepath_,
const uint64_t minfreespace_,
std::string *branch_)
{
int rv;
Value *v;
uint64_t now;
string branch;
if(timeout == 0)
return func_(branches_,fusepath_,minfreespace_,branch_);
now = local::get_time();
pthread_mutex_lock(&_lock);
v = &_cache[fusepath_];
if((now - v->time) >= timeout)
{
pthread_mutex_unlock(&_lock);
rv = func_(branches_,fusepath_,minfreespace_,&branch);
if(rv == -1)
return -1;
pthread_mutex_lock(&_lock);
v->time = now;
v->path = branch;
}
*branch_ = v->path;
pthread_mutex_unlock(&_lock);
return 0;
}

61
src/policy_cache.hpp

@ -0,0 +1,61 @@
/*
ISC License
Copyright (c) 2019, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#pragma once
#include "policy.hpp"
#include <string>
#include <map>
#include <pthread.h>
#include <stdint.h>
class PolicyCache
{
public:
struct Value
{
Value();
uint64_t time;
std::string path;
};
public:
PolicyCache(void);
public:
void erase(const char *fusepath_);
void cleanup(const int prob_ = 1);
void clear(void);
public:
int operator()(Policy::Func::Search &func_,
const Branches &branches_,
const char *fusepath_,
const uint64_t minfreespace_,
std::string *branch_);
public:
uint64_t timeout;
private:
pthread_mutex_t _lock;
std::map<std::string,Value> _cache;
};

35
src/policy_epall.cpp

@ -128,24 +128,21 @@ namespace epall
} }
} }
namespace mergerfs
int
Policy::Func::epall(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::epall(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
switch(type)
{
case Category::Enum::create:
return epall::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return epall::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return epall::search(branches_,fusepath,paths);
}
}
switch(type)
{
case Category::Enum::create:
return epall::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return epall::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return epall::search(branches_,fusepath,paths);
}
} }

36
src/policy_epff.cpp

@ -27,7 +27,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace epff namespace epff
{ {
@ -126,24 +125,21 @@ namespace epff
} }
} }
namespace mergerfs
int
Policy::Func::epff(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::epff(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
switch(type)
{
case Category::Enum::create:
return epff::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return epff::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return epff::search(branches_,fusepath,paths);
}
}
switch(type)
{
case Category::Enum::create:
return epff::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return epff::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return epff::search(branches_,fusepath,paths);
}
} }

36
src/policy_eplfs.cpp

@ -28,7 +28,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace eplfs namespace eplfs
{ {
@ -162,24 +161,21 @@ namespace eplfs
} }
} }
namespace mergerfs
int
Policy::Func::eplfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::eplfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
switch(type)
{
case Category::Enum::create:
return eplfs::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return eplfs::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return eplfs::search(branches_,fusepath,paths);
}
}
switch(type)
{
case Category::Enum::create:
return eplfs::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return eplfs::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return eplfs::search(branches_,fusepath,paths);
}
} }

36
src/policy_eplus.cpp

@ -28,7 +28,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace eplus namespace eplus
{ {
@ -162,24 +161,21 @@ namespace eplus
} }
} }
namespace mergerfs
int
Policy::Func::eplus(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::eplus(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
switch(type)
{
case Category::Enum::create:
return eplus::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return eplus::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return eplus::search(branches_,fusepath,paths);
}
}
switch(type)
{
case Category::Enum::create:
return eplus::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return eplus::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return eplus::search(branches_,fusepath,paths);
}
} }

36
src/policy_epmfs.cpp

@ -28,7 +28,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace epmfs namespace epmfs
{ {
@ -162,24 +161,21 @@ namespace epmfs
} }
} }
namespace mergerfs
int
Policy::Func::epmfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::epmfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
switch(type)
{
case Category::Enum::create:
return epmfs::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return epmfs::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return epmfs::search(branches_,fusepath,paths);
}
}
switch(type)
{
case Category::Enum::create:
return epmfs::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return epmfs::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return epmfs::search(branches_,fusepath,paths);
}
} }

29
src/policy_eprand.cpp

@ -24,21 +24,18 @@
using std::string; using std::string;
using std::vector; using std::vector;
namespace mergerfs
int
Policy::Func::eprand(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::eprand(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
int rv;
rv = Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
if(rv == 0)
std::random_shuffle(paths.begin(),paths.end());
return rv;
}
int rv;
rv = Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
if(rv == 0)
std::random_shuffle(paths.begin(),paths.end());
return rv;
} }

17
src/policy_erofs.cpp

@ -23,15 +23,12 @@
using std::string; using std::string;
using std::vector; using std::vector;
namespace mergerfs
int
Policy::Func::erofs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::erofs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
return (errno=EROFS,-1);
}
return (errno=EROFS,-1);
} }

21
src/policy_ff.cpp

@ -65,18 +65,15 @@ namespace ff
} }
} }
namespace mergerfs
int
Policy::Func::ff(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::ff(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
if(type == Category::Enum::create)
return ff::create(branches_,minfreespace,paths);
if(type == Category::Enum::create)
return ff::create(branches_,minfreespace,paths);
return Policy::Func::epff(type,branches_,fusepath,minfreespace,paths);
}
return Policy::Func::epff(type,branches_,fusepath,minfreespace,paths);
} }

17
src/policy_invalid.cpp

@ -23,15 +23,12 @@
using std::string; using std::string;
using std::vector; using std::vector;
namespace mergerfs
int
Policy::Func::invalid(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::invalid(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
return (errno=EINVAL,-1);
}
return (errno=EINVAL,-1);
} }

22
src/policy_lfs.cpp

@ -28,7 +28,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace lfs namespace lfs
{ {
@ -77,18 +76,15 @@ namespace lfs
} }
} }
namespace mergerfs
int
Policy::Func::lfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::lfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
if(type == Category::Enum::create)
return lfs::create(branches_,minfreespace,paths);
if(type == Category::Enum::create)
return lfs::create(branches_,minfreespace,paths);
return Policy::Func::eplfs(type,branches_,fusepath,minfreespace,paths);
}
return Policy::Func::eplfs(type,branches_,fusepath,minfreespace,paths);
} }

22
src/policy_lus.cpp

@ -28,7 +28,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace lus namespace lus
{ {
@ -77,18 +76,15 @@ namespace lus
} }
} }
namespace mergerfs
int
Policy::Func::lus(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::lus(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
if(type == Category::Enum::create)
return lus::create(branches_,minfreespace,paths);
if(type == Category::Enum::create)
return lus::create(branches_,minfreespace,paths);
return Policy::Func::eplus(type,branches_,fusepath,minfreespace,paths);
}
return Policy::Func::eplus(type,branches_,fusepath,minfreespace,paths);
} }

22
src/policy_mfs.cpp

@ -27,7 +27,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Category;
namespace mfs namespace mfs
{ {
@ -76,18 +75,15 @@ namespace mfs
} }
} }
namespace mergerfs
int
Policy::Func::mfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::mfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
if(type == Category::Enum::create)
return mfs::create(branches_,minfreespace,paths);
if(type == Category::Enum::create)
return mfs::create(branches_,minfreespace,paths);
return Policy::Func::epmfs(type,branches_,fusepath,minfreespace,paths);
}
return Policy::Func::epmfs(type,branches_,fusepath,minfreespace,paths);
} }

35
src/policy_newest.cpp

@ -161,24 +161,21 @@ namespace newest
} }
} }
namespace mergerfs
int
Policy::Func::newest(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::newest(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
switch(type)
{
case Category::Enum::create:
return newest::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return newest::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return newest::search(branches_,fusepath,paths);
}
}
switch(type)
{
case Category::Enum::create:
return newest::create(branches_,fusepath,minfreespace,paths);
case Category::Enum::action:
return newest::action(branches_,fusepath,paths);
case Category::Enum::search:
default:
return newest::search(branches_,fusepath,paths);
}
} }

29
src/policy_rand.cpp

@ -24,21 +24,18 @@
using std::string; using std::string;
using std::vector; using std::vector;
namespace mergerfs
int
Policy::Func::rand(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{ {
int
Policy::Func::rand(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
int rv;
rv = Policy::Func::all(type,branches_,fusepath,minfreespace,paths);
if(rv == 0)
std::random_shuffle(paths.begin(),paths.end());
return rv;
}
int rv;
rv = Policy::Func::all(type,branches_,fusepath,minfreespace,paths);
if(rv == 0)
std::random_shuffle(paths.begin(),paths.end());
return rv;
} }

1
src/readlink.cpp

@ -29,7 +29,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

49
src/release.cpp

@ -14,34 +14,37 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/ */
#include <fuse.h>
#include <string>
#include "config.hpp" #include "config.hpp"
#include "errno.hpp" #include "errno.hpp"
#include "fileinfo.hpp" #include "fileinfo.hpp"
#include "fs_base_close.hpp" #include "fs_base_close.hpp"
#include "fs_base_fadvise.hpp" #include "fs_base_fadvise.hpp"
static
int
_release(FileInfo *fi,
const bool dropcacheonclose)
#include <fuse.h>
#include <string>
namespace local
{ {
// according to Feh of nocache calling it once doesn't always work
// https://github.com/Feh/nocache
if(dropcacheonclose)
{
fs::fadvise_dontneed(fi->fd);
fs::fadvise_dontneed(fi->fd);
}
static
int
release(FileInfo *fi_,
const bool dropcacheonclose_)
{
// according to Feh of nocache calling it once doesn't always work
// https://github.com/Feh/nocache
if(dropcacheonclose_)
{
fs::fadvise_dontneed(fi_->fd);
fs::fadvise_dontneed(fi_->fd);
}
fs::close(fi->fd);
fs::close(fi_->fd);
delete fi;
delete fi_;
return 0;
return 0;
}
} }
namespace mergerfs namespace mergerfs
@ -49,13 +52,15 @@ namespace mergerfs
namespace fuse namespace fuse
{ {
int int
release(const char *fusepath,
fuse_file_info *ffi)
release(const char *fusepath_,
fuse_file_info *ffi_)
{ {
const Config &config = Config::get(); const Config &config = Config::get();
FileInfo *fi = reinterpret_cast<FileInfo*>(ffi->fh);
FileInfo *fi = reinterpret_cast<FileInfo*>(ffi_->fh);
config.open_cache.cleanup(10);
return _release(fi,config.dropcacheonclose);
return local::release(fi,config.dropcacheonclose);
} }
} }
} }

29
src/releasedir.cpp

@ -14,17 +14,21 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/ */
#include <fuse.h>
#include "config.hpp"
#include "dirinfo.hpp" #include "dirinfo.hpp"
static
int
_releasedir(DirInfo *di)
#include <fuse.h>
namespace local
{ {
delete di;
static
int
releasedir(DirInfo *di_)
{
delete di_;
return 0;
return 0;
}
} }
namespace mergerfs namespace mergerfs
@ -32,12 +36,15 @@ namespace mergerfs
namespace fuse namespace fuse
{ {
int int
releasedir(const char *fusepath,
fuse_file_info *ffi)
releasedir(const char *fusepath_,
fuse_file_info *ffi_)
{ {
DirInfo *di = reinterpret_cast<DirInfo*>(ffi->fh);
const Config &config = Config::get();
DirInfo *di = reinterpret_cast<DirInfo*>(ffi_->fh);
config.open_cache.cleanup(10);
return ::_releasedir(di);
return local::releasedir(di);
} }
} }
} }

1
src/removexattr.cpp

@ -29,7 +29,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

2
src/rename.cpp

@ -283,6 +283,8 @@ namespace mergerfs
const ugid::Set ugid(fc->uid,fc->gid); const ugid::Set ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.branches_lock); const rwlock::ReadGuard readlock(&config.branches_lock);
config.open_cache.erase(oldpath);
if(config.create->path_preserving() && !config.ignorepponrename) if(config.create->path_preserving() && !config.ignorepponrename)
return _rename_preserve_path(config.getattr, return _rename_preserve_path(config.getattr,
config.rename, config.rename,

1
src/rmdir.cpp

@ -30,7 +30,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

11
src/setxattr.cpp

@ -37,7 +37,6 @@ static const char SECURITY_CAPABILITY[] = "security.capability";
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
using mergerfs::FuseFunc; using mergerfs::FuseFunc;
using namespace mergerfs; using namespace mergerfs;
@ -229,6 +228,9 @@ _setxattr_controlfile_func_policy(Config &config,
if((flags & XATTR_CREATE) == XATTR_CREATE) if((flags & XATTR_CREATE) == XATTR_CREATE)
return -EEXIST; return -EEXIST;
if(funcname == "open")
config.open_cache.clear();
rv = config.set_func_policy(funcname,attrval); rv = config.set_func_policy(funcname,attrval);
if(rv == -1) if(rv == -1)
return -errno; return -errno;
@ -248,6 +250,9 @@ _setxattr_controlfile_category_policy(Config &config,
if((flags & XATTR_CREATE) == XATTR_CREATE) if((flags & XATTR_CREATE) == XATTR_CREATE)
return -EEXIST; return -EEXIST;
if(categoryname == "search")
config.open_cache.clear();
rv = config.set_category_policy(categoryname,attrval); rv = config.set_category_policy(categoryname,attrval);
if(rv == -1) if(rv == -1)
return -errno; return -errno;
@ -340,6 +345,10 @@ _setxattr_controlfile(Config &config,
attr[3], attr[3],
attrval, attrval,
flags); flags);
else if((attr[2] == "cache") && (attr[3] == "open"))
return _setxattr_uint64_t(attrval,
flags,
config.open_cache.timeout);
break; break;
default: default:

1
src/truncate.cpp

@ -32,7 +32,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

106
src/unlink.cpp

@ -1,5 +1,5 @@
/* /*
Copyright (c) 2016, Antonio SJ Musumeci <trapexit@spawn.link>
Copyright (c) 2019, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above purpose with or without fee is hereby granted, provided that the above
@ -14,13 +14,6 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/ */
#include <fuse.h>
#include <unistd.h>
#include <string>
#include <vector>
#include "config.hpp" #include "config.hpp"
#include "errno.hpp" #include "errno.hpp"
#include "fs_base_unlink.hpp" #include "fs_base_unlink.hpp"
@ -29,57 +22,66 @@
#include "rwlock.hpp" #include "rwlock.hpp"
#include "ugid.hpp" #include "ugid.hpp"
#include <fuse.h>
#include <string>
#include <vector>
#include <unistd.h>
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static
int
_unlink_loop_core(const string *basepath,
const char *fusepath,
const int error)
namespace local
{ {
int rv;
string fullpath;
static
int
unlink_loop_core(const string *basepath_,
const char *fusepath_,
const int error_)
{
int rv;
string fullpath;
fs::path::make(basepath,fusepath,fullpath);
fs::path::make(basepath_,fusepath_,fullpath);
rv = fs::unlink(fullpath);
rv = fs::unlink(fullpath);
return error::calc(rv,error,errno);
}
return error::calc(rv,error_,errno);
}
static
int
_unlink_loop(const vector<const string*> &basepaths,
const char *fusepath)
{
int error;
static
int
unlink_loop(const vector<const string*> &basepaths_,
const char *fusepath_)
{
int error;
error = -1;
for(size_t i = 0, ei = basepaths.size(); i != ei; i++)
{
error = _unlink_loop_core(basepaths[i],fusepath,error);
}
error = -1;
for(size_t i = 0, ei = basepaths_.size(); i != ei; i++)
{
error = local::unlink_loop_core(basepaths_[i],fusepath_,error);
}
return -error;
}
return -error;
}
static
int
_unlink(Policy::Func::Action actionFunc,
const Branches &branches_,
const uint64_t minfreespace,
const char *fusepath)
{
int rv;
vector<const string*> basepaths;
static
int
unlink(Policy::Func::Action actionFunc_,
const Branches &branches_,
const uint64_t minfreespace_,
const char *fusepath_)
{
int rv;
vector<const string*> basepaths;
rv = actionFunc(branches_,fusepath,minfreespace,basepaths);
if(rv == -1)
return -errno;
rv = actionFunc_(branches_,fusepath_,minfreespace_,basepaths);
if(rv == -1)
return -errno;
return _unlink_loop(basepaths,fusepath);
return local::unlink_loop(basepaths,fusepath_);
}
} }
namespace mergerfs namespace mergerfs
@ -87,17 +89,19 @@ namespace mergerfs
namespace fuse namespace fuse
{ {
int int
unlink(const char *fusepath)
unlink(const char *fusepath_)
{ {
const fuse_context *fc = fuse_get_context(); const fuse_context *fc = fuse_get_context();
const Config &config = Config::get(fc); const Config &config = Config::get(fc);
const ugid::Set ugid(fc->uid,fc->gid); const ugid::Set ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.branches_lock); const rwlock::ReadGuard readlock(&config.branches_lock);
return _unlink(config.unlink,
config.branches,
config.minfreespace,
fusepath);
config.open_cache.erase(fusepath_);
return local::unlink(config.unlink,
config.branches,
config.minfreespace,
fusepath_);
} }
} }
} }

1
src/utimens.cpp

@ -31,7 +31,6 @@
using std::string; using std::string;
using std::vector; using std::vector;
using mergerfs::Policy;
static static
int int

Loading…
Cancel
Save