Browse Source

add policy cache for 'open'

A fusepath -> basepath cache for `open` to limit the overhead of FUSE in 'open, read/write, close' patterns (such as Transmission).
pull/565/head
Antonio SJ Musumeci 6 years ago
parent
commit
7a057daa0c
  1. 10
      README.md
  2. 2
      libfuse/configure.ac
  3. 20
      man/mergerfs.1
  4. 2
      src/access.cpp
  5. 29
      src/category.cpp
  6. 19
      src/category.hpp
  7. 1
      src/chmod.cpp
  8. 1
      src/chown.cpp
  9. 4
      src/config.hpp
  10. 1
      src/getattr.cpp
  11. 2
      src/getxattr.cpp
  12. 1
      src/link.cpp
  13. 22
      src/listxattr.cpp
  14. 12
      src/open.cpp
  15. 16
      src/option_parser.cpp
  16. 54
      src/policy.cpp
  17. 32
      src/policy.hpp
  18. 9
      src/policy_all.cpp
  19. 128
      src/policy_cache.cpp
  20. 61
      src/policy_cache.hpp
  21. 9
      src/policy_epall.cpp
  22. 10
      src/policy_epff.cpp
  23. 10
      src/policy_eplfs.cpp
  24. 10
      src/policy_eplus.cpp
  25. 10
      src/policy_epmfs.cpp
  26. 9
      src/policy_eprand.cpp
  27. 9
      src/policy_erofs.cpp
  28. 9
      src/policy_ff.cpp
  29. 9
      src/policy_invalid.cpp
  30. 10
      src/policy_lfs.cpp
  31. 10
      src/policy_lus.cpp
  32. 10
      src/policy_mfs.cpp
  33. 9
      src/policy_newest.cpp
  34. 9
      src/policy_rand.cpp
  35. 1
      src/readlink.cpp
  36. 39
      src/release.cpp
  37. 27
      src/releasedir.cpp
  38. 1
      src/removexattr.cpp
  39. 2
      src/rename.cpp
  40. 1
      src/rmdir.cpp
  41. 11
      src/setxattr.cpp
  42. 1
      src/truncate.cpp
  43. 76
      src/unlink.cpp
  44. 1
      src/utimens.cpp

10
README.md

@ -84,6 +84,7 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs**
* **fsname=name**: sets the name of the filesystem as seen in **mount**, **df**, etc. Defaults to a list of the source paths concatenated together with the longest common prefix removed.
* **func.<func>=<policy>**: sets the specific FUSE function's policy. See below for the list of value types. Example: **func.getattr=newest**
* **category.<category>=<policy>**: Sets policy of all FUSE functions in the provided category. Example: **category.create=mfs**
* **cache.open=<int>**: 'open' policy cache timeout in seconds. (default: 0)
**NOTE:** Options are evaluated in the order listed so if the options are **func.rmdir=rand,category.action=ff** the **action** category setting will override the **rmdir** setting.
@ -491,6 +492,15 @@ It's a difficult balance between memory usage, cache bloat & duplication, and pe
Given the relatively high cost of FUSE due to the kernel <-> userspace round trips there are kernel side caches for file entries and attributes. The entry cache limits the `lookup` calls to mergerfs which ask if a file exists. The attribute cache limits the need to make `getattr` calls to mergerfs which provide file attributes (mode, size, type, etc.). As with the page cache these should not be used if the underlying filesystems are being manipulated at the same time as it could lead to odd behavior or data corruption. The options for setting these are `entry_timeout` and `negative_timeout` for the entry cache and `attr_timeout` for the attributes cache. `negative_timeout` refers to the timeout for negative responses to lookups (non-existant files).
#### policy caching
Policies are run every time a function is called. These policies can be expensive depending on the setup and usage patterns. Generally we wouldn't want to cache policy results because it may result in stale responses if the underlying drives are used directly.
The `open` policy cache will cache the result of an `open` policy for a particular input for `cache.open` seconds or until the file is unlinked. Each file close (release) will randomly chose to clean up the cache of expired entries.
This cache is useful in cases like that of **Transmission** which has a "open, read/write, close" pattern (which is much more costly due to the FUSE overhead than normal.)
#### writeback caching
writeback caching is a technique for improving write speeds by batching writes at a faster device and then bulk writing to the slower device. With FUSE the kernel will wait for a number of writes to be made and then send it to the filesystem as one request. mergerfs currently uses a slightly modified and vendored libfuse 2.9.7 which does not support writeback caching. However, a prototype port to libfuse 3.x has been made and the writeback cache appears to work as expected (though performance improvements greatly depend on the way the client app writes data). Once the port is complete and thoroughly tested writeback caching will be available.

2
libfuse/configure.ac

@ -1,4 +1,4 @@
AC_INIT(fuse, 2.9.8-mergerfs)
AC_INIT(fuse, 2.9.7-mergerfs_2.26.0)
AC_PREREQ(2.59d)
AC_CONFIG_MACRO_DIR([m4])

20
man/mergerfs.1

@ -211,6 +211,9 @@ Example: \f[B]func.getattr=newest\f[]
\f[B]category.<category>=<policy>\f[]: Sets policy of all FUSE functions
in the provided category.
Example: \f[B]category.create=mfs\f[]
.IP \[bu] 2
\f[B]cache.open=\f[]: \[aq]open\[aq] policy cache timeout in seconds.
(default: 0)
.PP
\f[B]NOTE:\f[] Options are evaluated in the order listed so if the
options are \f[B]func.rmdir=rand,category.action=ff\f[] the
@ -1034,6 +1037,23 @@ The options for setting these are \f[C]entry_timeout\f[] and
for the attributes cache.
\f[C]negative_timeout\f[] refers to the timeout for negative responses
to lookups (non\-existant files).
.SS policy caching
.PP
Policies are run every time a function is called.
These policies can be expensive depending on the setup and usage
patterns.
Generally we wouldn\[aq]t want to cache policy results because it may
result in stale responses if the underlying drives are used directly.
.PP
The \f[C]open\f[] policy cache will cache the result of an \f[C]open\f[]
policy for a particular input for \f[C]cache.open\f[] seconds or until
the file is unlinked.
Each file close (release) will randomly chose to clean up the cache of
expired entries.
.PP
This cache is useful in cases like that of \f[B]Transmission\f[] which
has a "open, read/write, close" pattern (which is much more costly due
to the FUSE overhead than normal.)
.SS writeback caching
.PP
writeback caching is a technique for improving write speeds by batching

2
src/access.cpp

@ -26,8 +26,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
using mergerfs::Category;
static
int

29
src/category.cpp

@ -22,25 +22,23 @@
#define CATEGORY(X) Category(Category::Enum::X,#X)
namespace mergerfs
{
const std::vector<Category> Category::_categories_ =
const std::vector<Category> Category::_categories_ =
buildvector<Category,true>
(CATEGORY(invalid))
(CATEGORY(action))
(CATEGORY(create))
(CATEGORY(search));
const Category * const Category::categories = &_categories_[1];
const Category * const Category::categories = &_categories_[1];
const Category &Category::invalid = Category::categories[Category::Enum::invalid];
const Category &Category::action = Category::categories[Category::Enum::action];
const Category &Category::create = Category::categories[Category::Enum::create];
const Category &Category::search = Category::categories[Category::Enum::search];
const Category &Category::invalid = Category::categories[Category::Enum::invalid];
const Category &Category::action = Category::categories[Category::Enum::action];
const Category &Category::create = Category::categories[Category::Enum::create];
const Category &Category::search = Category::categories[Category::Enum::search];
const Category&
Category::find(const std::string &str)
{
const Category&
Category::find(const std::string &str)
{
for(int i = Enum::BEGIN; i != Enum::END; ++i)
{
if(categories[i] == str)
@ -48,15 +46,14 @@ namespace mergerfs
}
return invalid;
}
}
const Category&
Category::find(const Category::Enum::Type i)
{
const Category&
Category::find(const Category::Enum::Type i)
{
if(i >= Category::Enum::BEGIN &&
i < Category::Enum::END)
return categories[i];
return invalid;
}
}

19
src/category.hpp

@ -19,11 +19,9 @@
#include <string>
#include <vector>
namespace mergerfs
class Category
{
class Category
{
public:
public:
struct Enum
{
enum Type
@ -37,11 +35,11 @@ namespace mergerfs
};
};
private:
private:
Enum::Type _enum;
std::string _str;
public:
public:
Category()
: _enum(invalid),
_str(invalid)
@ -55,7 +53,7 @@ namespace mergerfs
{
}
public:
public:
operator const Enum::Type() const { return _enum; }
operator const std::string&() const { return _str; }
operator const Category*() const { return this; }
@ -72,16 +70,15 @@ namespace mergerfs
bool operator<(const Category &r) const
{ return _enum < r._enum; }
public:
public:
static const Category &find(const std::string&);
static const Category &find(const Enum::Type);
public:
public:
static const std::vector<Category> _categories_;
static const Category * const categories;
static const Category &invalid;
static const Category &action;
static const Category &create;
static const Category &search;
};
}
};

1
src/chmod.cpp

@ -29,7 +29,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

1
src/chown.cpp

@ -29,7 +29,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
using mergerfs::Config;
static

4
src/config.hpp

@ -19,6 +19,7 @@
#include "branch.hpp"
#include "fusefunc.hpp"
#include "policy.hpp"
#include "policy_cache.hpp"
#include <fuse.h>
@ -102,6 +103,9 @@ namespace mergerfs
const Policy *&unlink;
const Policy *&utimens;
public:
mutable PolicyCache open_cache;
public:
const std::string controlfile;

1
src/getattr.cpp

@ -30,7 +30,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

2
src/getxattr.cpp

@ -306,6 +306,8 @@ _getxattr_controlfile(const Config &config,
_getxattr_controlfile_category_policy(config,attr[3],attrvalue);
else if(attr[2] == "func")
_getxattr_controlfile_fusefunc_policy(config,attr[3],attrvalue);
else if((attr[2] == "cache") && (attr[3] == "open"))
_getxattr_controlfile_uint64_t(config.open_cache.timeout,attrvalue);
break;
}

1
src/link.cpp

@ -30,7 +30,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
using namespace mergerfs;
static

22
src/listxattr.cpp

@ -43,24 +43,26 @@ _listxattr_controlfile(char *list,
string xattrs;
const vector<string> strs =
buildvector<string>
("user.mergerfs.srcmounts")
("user.mergerfs.branches")
("user.mergerfs.minfreespace")
("user.mergerfs.moveonenospc")
("user.mergerfs.cache.open")
("user.mergerfs.direct_io")
("user.mergerfs.dropcacheonclose")
("user.mergerfs.symlinkify")
("user.mergerfs.symlinkify_timeout")
("user.mergerfs.nullrw")
("user.mergerfs.ignorepponrename")
("user.mergerfs.link_cow")
("user.mergerfs.minfreespace")
("user.mergerfs.moveonenospc")
("user.mergerfs.nullrw")
("user.mergerfs.pid")
("user.mergerfs.policies")
("user.mergerfs.security_capability")
("user.mergerfs.xattr")
("user.mergerfs.srcmounts")
("user.mergerfs.statfs")
("user.mergerfs.statfs_ignore")
("user.mergerfs.direct_io")
("user.mergerfs.policies")
("user.mergerfs.symlinkify")
("user.mergerfs.symlinkify_timeout")
("user.mergerfs.version")
("user.mergerfs.pid");
("user.mergerfs.xattr")
;
xattrs.reserve(1024);
for(size_t i = 0; i < strs.size(); i++)

12
src/open.cpp

@ -20,6 +20,7 @@
#include "fs_base_open.hpp"
#include "fs_cow.hpp"
#include "fs_path.hpp"
#include "policy_cache.hpp"
#include "rwlock.hpp"
#include "ugid.hpp"
@ -28,11 +29,8 @@
#include <string>
#include <vector>
#include <fcntl.h>
using std::string;
using std::vector;
using mergerfs::Policy;
namespace local
{
@ -64,6 +62,7 @@ namespace local
static
int
open(Policy::Func::Search searchFunc_,
PolicyCache &cache,
const Branches &branches_,
const uint64_t minfreespace_,
const char *fusepath_,
@ -72,13 +71,13 @@ namespace local
uint64_t *fh_)
{
int rv;
vector<const string*> basepaths;
string basepath;
rv = searchFunc_(branches_,fusepath_,minfreespace_,basepaths);
rv = cache(searchFunc_,branches_,fusepath_,minfreespace_,&basepath);
if(rv == -1)
return -errno;
return local::open_core(*basepaths[0],fusepath_,flags_,link_cow_,fh_);
return local::open_core(basepath,fusepath_,flags_,link_cow_,fh_);
}
}
@ -97,6 +96,7 @@ namespace mergerfs
ffi_->direct_io = config.direct_io;
return local::open(config.open,
config.open_cache,
config.branches,
config.minfreespace,
fusepath_,

16
src/option_parser.cpp

@ -202,6 +202,18 @@ parse_and_process_statfsignore(const std::string &value_,
return 0;
}
static
int
parse_and_process_cache(Config &config_,
const string &func_,
const string &value_)
{
if(func_ == "open")
return parse_and_process(value_,config_.open_cache.timeout);
return 1;
}
static
int
parse_and_process_arg(Config &config,
@ -233,6 +245,8 @@ parse_and_process_kv_arg(Config &config,
rv = config.set_func_policy(keypart[1],value);
else if(keypart[0] == "category")
rv = config.set_category_policy(keypart[1],value);
else if(keypart[0] == "cache")
rv = parse_and_process_cache(config,keypart[1],value);
}
else
{
@ -336,6 +350,8 @@ usage(void)
" splice_write, splice_move\n"
" -o func.<f>=<p> Set function <f> to policy <p>\n"
" -o category.<c>=<p> Set functions in category <c> to <p>\n"
" -o cache.open=<int> 'open' policy cache timeout in seconds.\n"
" default = 0 (disabled)\n"
" -o direct_io Bypass page caching, may increase write\n"
" speeds at the cost of reads. Please read docs\n"
" for more details as there are tradeoffs.\n"

54
src/policy.cpp

@ -24,9 +24,7 @@
#define PRESERVES_PATH true
#define DOESNT_PRESERVE_PATH false
namespace mergerfs
{
const std::vector<Policy> Policy::_policies_ =
const std::vector<Policy> Policy::_policies_ =
buildvector<Policy,true>
(POLICY(invalid,DOESNT_PRESERVE_PATH))
(POLICY(all,DOESNT_PRESERVE_PATH))
@ -44,45 +42,45 @@ namespace mergerfs
(POLICY(newest,DOESNT_PRESERVE_PATH))
(POLICY(rand,DOESNT_PRESERVE_PATH));
const Policy * const Policy::policies = &_policies_[1];
const Policy * const Policy::policies = &_policies_[1];
#define CONST_POLICY(X) const Policy &Policy::X = Policy::policies[Policy::Enum::X]
CONST_POLICY(invalid);
CONST_POLICY(all);
CONST_POLICY(epall);
CONST_POLICY(epff);
CONST_POLICY(eplfs);
CONST_POLICY(eplus);
CONST_POLICY(epmfs);
CONST_POLICY(eprand);
CONST_POLICY(erofs);
CONST_POLICY(ff);
CONST_POLICY(lfs);
CONST_POLICY(lus);
CONST_POLICY(mfs);
CONST_POLICY(newest);
CONST_POLICY(rand);
CONST_POLICY(invalid);
CONST_POLICY(all);
CONST_POLICY(epall);
CONST_POLICY(epff);
CONST_POLICY(eplfs);
CONST_POLICY(eplus);
CONST_POLICY(epmfs);
CONST_POLICY(eprand);
CONST_POLICY(erofs);
CONST_POLICY(ff);
CONST_POLICY(lfs);
CONST_POLICY(lus);
CONST_POLICY(mfs);
CONST_POLICY(newest);
CONST_POLICY(rand);
const Policy&
Policy::find(const std::string &str)
{
for(int i = Enum::BEGIN; i != Enum::END; ++i)
const Policy&
Policy::find(const std::string &str)
{
for(int i = Enum::BEGIN; i != Enum::END; ++i)
{
if(policies[i] == str)
return policies[i];
}
return invalid;
}
}
const Policy&
Policy::find(const Policy::Enum::Type i)
{
const Policy&
Policy::find(const Policy::Enum::Type i)
{
if(i >= Policy::Enum::BEGIN &&
i < Policy::Enum::END)
return policies[i];
return invalid;
}
}

32
src/policy.hpp

@ -24,11 +24,9 @@
#include <string>
#include <vector>
namespace mergerfs
class Policy
{
class Policy
{
public:
public:
struct Enum
{
enum Type
@ -88,6 +86,19 @@ namespace mergerfs
return func(T,b,c.c_str(),d,e);
}
int
operator()(const Branches &b,const char *c,cuint64_t d,string *e)
{
int rv;
cstrptrvec v;
rv = func(T,b,c,d,v);
if(!v.empty())
*e = *v[0];
return rv;
}
private:
const Ptr func;
};
@ -113,13 +124,13 @@ namespace mergerfs
static int rand(CType,const Branches&,const char *,cuint64_t,cstrptrvec&);
};
private:
private:
Enum::Type _enum;
std::string _str;
Func::Ptr _func;
bool _path_preserving;
public:
public:
Policy()
: _enum(invalid),
_str(invalid),
@ -145,7 +156,7 @@ namespace mergerfs
return _path_preserving;
}
public:
public:
operator const Enum::Type() const { return _enum; }
operator const std::string&() const { return _str; }
operator const Func::Ptr() const { return _func; }
@ -164,11 +175,11 @@ namespace mergerfs
bool operator<(const Policy &r) const
{ return _enum < r._enum; }
public:
public:
static const Policy &find(const std::string&);
static const Policy &find(const Enum::Type);
public:
public:
static const std::vector<Policy> _policies_;
static const Policy * const policies;
@ -187,5 +198,4 @@ namespace mergerfs
static const Policy &mfs;
static const Policy &newest;
static const Policy &rand;
};
}
};

9
src/policy_all.cpp

@ -65,18 +65,15 @@ namespace all
}
}
namespace mergerfs
{
int
Policy::Func::all(const Category::Enum::Type type,
int
Policy::Func::all(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
if(type == Category::Enum::create)
return all::create(branches_,minfreespace,paths);
return Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
}
}

128
src/policy_cache.cpp

@ -0,0 +1,128 @@
#include "policy_cache.hpp"
#include <cstdlib>
#include <map>
#include <string>
#include <vector>
#include <sys/time.h>
using std::map;
using std::string;
using std::vector;
static const uint64_t DEFAULT_TIMEOUT = 0;
namespace local
{
static
uint64_t
get_time(void)
{
uint64_t rv;
struct timeval now;
::gettimeofday(&now,NULL);
rv = now.tv_sec;
return rv;
}
}
PolicyCache::Value::Value()
: time(0),
path()
{
}
PolicyCache::PolicyCache(void)
: timeout(DEFAULT_TIMEOUT)
{
}
void
PolicyCache::erase(const char *fusepath_)
{
pthread_mutex_lock(&_lock);
_cache.erase(fusepath_);
pthread_mutex_unlock(&_lock);
}
void
PolicyCache::cleanup(const int prob_)
{
uint64_t now;
map<string,Value>::iterator i;
if(rand() % prob_)
return;
now = local::get_time();
pthread_mutex_lock(&_lock);
i = _cache.begin();
while(i != _cache.end())
{
if((now - i->second.time) >= timeout)
_cache.erase(i++);
else
++i;
}
pthread_mutex_unlock(&_lock);
}
void
PolicyCache::clear(void)
{
pthread_mutex_lock(&_lock);
_cache.clear();
pthread_mutex_unlock(&_lock);
}
int
PolicyCache::operator()(Policy::Func::Search &func_,
const Branches &branches_,
const char *fusepath_,
const uint64_t minfreespace_,
std::string *branch_)
{
int rv;
Value *v;
uint64_t now;
string branch;
if(timeout == 0)
return func_(branches_,fusepath_,minfreespace_,branch_);
now = local::get_time();
pthread_mutex_lock(&_lock);
v = &_cache[fusepath_];
if((now - v->time) >= timeout)
{
pthread_mutex_unlock(&_lock);
rv = func_(branches_,fusepath_,minfreespace_,&branch);
if(rv == -1)
return -1;
pthread_mutex_lock(&_lock);
v->time = now;
v->path = branch;
}
*branch_ = v->path;
pthread_mutex_unlock(&_lock);
return 0;
}

61
src/policy_cache.hpp

@ -0,0 +1,61 @@
/*
ISC License
Copyright (c) 2019, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#pragma once
#include "policy.hpp"
#include <string>
#include <map>
#include <pthread.h>
#include <stdint.h>
class PolicyCache
{
public:
struct Value
{
Value();
uint64_t time;
std::string path;
};
public:
PolicyCache(void);
public:
void erase(const char *fusepath_);
void cleanup(const int prob_ = 1);
void clear(void);
public:
int operator()(Policy::Func::Search &func_,
const Branches &branches_,
const char *fusepath_,
const uint64_t minfreespace_,
std::string *branch_);
public:
uint64_t timeout;
private:
pthread_mutex_t _lock;
std::map<std::string,Value> _cache;
};

9
src/policy_epall.cpp

@ -128,15 +128,13 @@ namespace epall
}
}
namespace mergerfs
{
int
Policy::Func::epall(const Category::Enum::Type type,
int
Policy::Func::epall(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
switch(type)
{
case Category::Enum::create:
@ -147,5 +145,4 @@ namespace mergerfs
default:
return epall::search(branches_,fusepath,paths);
}
}
}

10
src/policy_epff.cpp

@ -27,7 +27,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace epff
{
@ -126,15 +125,13 @@ namespace epff
}
}
namespace mergerfs
{
int
Policy::Func::epff(const Category::Enum::Type type,
int
Policy::Func::epff(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
switch(type)
{
case Category::Enum::create:
@ -145,5 +142,4 @@ namespace mergerfs
default:
return epff::search(branches_,fusepath,paths);
}
}
}

10
src/policy_eplfs.cpp

@ -28,7 +28,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace eplfs
{
@ -162,15 +161,13 @@ namespace eplfs
}
}
namespace mergerfs
{
int
Policy::Func::eplfs(const Category::Enum::Type type,
int
Policy::Func::eplfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
switch(type)
{
case Category::Enum::create:
@ -181,5 +178,4 @@ namespace mergerfs
default:
return eplfs::search(branches_,fusepath,paths);
}
}
}

10
src/policy_eplus.cpp

@ -28,7 +28,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace eplus
{
@ -162,15 +161,13 @@ namespace eplus
}
}
namespace mergerfs
{
int
Policy::Func::eplus(const Category::Enum::Type type,
int
Policy::Func::eplus(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
switch(type)
{
case Category::Enum::create:
@ -181,5 +178,4 @@ namespace mergerfs
default:
return eplus::search(branches_,fusepath,paths);
}
}
}

10
src/policy_epmfs.cpp

@ -28,7 +28,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace epmfs
{
@ -162,15 +161,13 @@ namespace epmfs
}
}
namespace mergerfs
{
int
Policy::Func::epmfs(const Category::Enum::Type type,
int
Policy::Func::epmfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
switch(type)
{
case Category::Enum::create:
@ -181,5 +178,4 @@ namespace mergerfs
default:
return epmfs::search(branches_,fusepath,paths);
}
}
}

9
src/policy_eprand.cpp

@ -24,15 +24,13 @@
using std::string;
using std::vector;
namespace mergerfs
{
int
Policy::Func::eprand(const Category::Enum::Type type,
int
Policy::Func::eprand(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
int rv;
rv = Policy::Func::epall(type,branches_,fusepath,minfreespace,paths);
@ -40,5 +38,4 @@ namespace mergerfs
std::random_shuffle(paths.begin(),paths.end());
return rv;
}
}

9
src/policy_erofs.cpp

@ -23,15 +23,12 @@
using std::string;
using std::vector;
namespace mergerfs
{
int
Policy::Func::erofs(const Category::Enum::Type type,
int
Policy::Func::erofs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
return (errno=EROFS,-1);
}
}

9
src/policy_ff.cpp

@ -65,18 +65,15 @@ namespace ff
}
}
namespace mergerfs
{
int
Policy::Func::ff(const Category::Enum::Type type,
int
Policy::Func::ff(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
if(type == Category::Enum::create)
return ff::create(branches_,minfreespace,paths);
return Policy::Func::epff(type,branches_,fusepath,minfreespace,paths);
}
}

9
src/policy_invalid.cpp

@ -23,15 +23,12 @@
using std::string;
using std::vector;
namespace mergerfs
{
int
Policy::Func::invalid(const Category::Enum::Type type,
int
Policy::Func::invalid(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
return (errno=EINVAL,-1);
}
}

10
src/policy_lfs.cpp

@ -28,7 +28,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace lfs
{
@ -77,18 +76,15 @@ namespace lfs
}
}
namespace mergerfs
{
int
Policy::Func::lfs(const Category::Enum::Type type,
int
Policy::Func::lfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
if(type == Category::Enum::create)
return lfs::create(branches_,minfreespace,paths);
return Policy::Func::eplfs(type,branches_,fusepath,minfreespace,paths);
}
}

10
src/policy_lus.cpp

@ -28,7 +28,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace lus
{
@ -77,18 +76,15 @@ namespace lus
}
}
namespace mergerfs
{
int
Policy::Func::lus(const Category::Enum::Type type,
int
Policy::Func::lus(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
if(type == Category::Enum::create)
return lus::create(branches_,minfreespace,paths);
return Policy::Func::eplus(type,branches_,fusepath,minfreespace,paths);
}
}

10
src/policy_mfs.cpp

@ -27,7 +27,6 @@
using std::string;
using std::vector;
using mergerfs::Category;
namespace mfs
{
@ -76,18 +75,15 @@ namespace mfs
}
}
namespace mergerfs
{
int
Policy::Func::mfs(const Category::Enum::Type type,
int
Policy::Func::mfs(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
if(type == Category::Enum::create)
return mfs::create(branches_,minfreespace,paths);
return Policy::Func::epmfs(type,branches_,fusepath,minfreespace,paths);
}
}

9
src/policy_newest.cpp

@ -161,15 +161,13 @@ namespace newest
}
}
namespace mergerfs
{
int
Policy::Func::newest(const Category::Enum::Type type,
int
Policy::Func::newest(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
switch(type)
{
case Category::Enum::create:
@ -180,5 +178,4 @@ namespace mergerfs
default:
return newest::search(branches_,fusepath,paths);
}
}
}

9
src/policy_rand.cpp

@ -24,15 +24,13 @@
using std::string;
using std::vector;
namespace mergerfs
{
int
Policy::Func::rand(const Category::Enum::Type type,
int
Policy::Func::rand(const Category::Enum::Type type,
const Branches &branches_,
const char *fusepath,
const uint64_t minfreespace,
vector<const string*> &paths)
{
{
int rv;
rv = Policy::Func::all(type,branches_,fusepath,minfreespace,paths);
@ -40,5 +38,4 @@ namespace mergerfs
std::random_shuffle(paths.begin(),paths.end());
return rv;
}
}

1
src/readlink.cpp

@ -29,7 +29,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

39
src/release.cpp

@ -14,34 +14,37 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <fuse.h>
#include <string>
#include "config.hpp"
#include "errno.hpp"
#include "fileinfo.hpp"
#include "fs_base_close.hpp"
#include "fs_base_fadvise.hpp"
static
int
_release(FileInfo *fi,
const bool dropcacheonclose)
#include <fuse.h>
#include <string>
namespace local
{
static
int
release(FileInfo *fi_,
const bool dropcacheonclose_)
{
// according to Feh of nocache calling it once doesn't always work
// https://github.com/Feh/nocache
if(dropcacheonclose)
if(dropcacheonclose_)
{
fs::fadvise_dontneed(fi->fd);
fs::fadvise_dontneed(fi->fd);
fs::fadvise_dontneed(fi_->fd);
fs::fadvise_dontneed(fi_->fd);
}
fs::close(fi->fd);
fs::close(fi_->fd);
delete fi;
delete fi_;
return 0;
}
}
namespace mergerfs
@ -49,13 +52,15 @@ namespace mergerfs
namespace fuse
{
int
release(const char *fusepath,
fuse_file_info *ffi)
release(const char *fusepath_,
fuse_file_info *ffi_)
{
const Config &config = Config::get();
FileInfo *fi = reinterpret_cast<FileInfo*>(ffi->fh);
FileInfo *fi = reinterpret_cast<FileInfo*>(ffi_->fh);
config.open_cache.cleanup(10);
return _release(fi,config.dropcacheonclose);
return local::release(fi,config.dropcacheonclose);
}
}
}

27
src/releasedir.cpp

@ -14,17 +14,21 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <fuse.h>
#include "config.hpp"
#include "dirinfo.hpp"
static
int
_releasedir(DirInfo *di)
#include <fuse.h>
namespace local
{
delete di;
static
int
releasedir(DirInfo *di_)
{
delete di_;
return 0;
}
}
namespace mergerfs
@ -32,12 +36,15 @@ namespace mergerfs
namespace fuse
{
int
releasedir(const char *fusepath,
fuse_file_info *ffi)
releasedir(const char *fusepath_,
fuse_file_info *ffi_)
{
DirInfo *di = reinterpret_cast<DirInfo*>(ffi->fh);
const Config &config = Config::get();
DirInfo *di = reinterpret_cast<DirInfo*>(ffi_->fh);
config.open_cache.cleanup(10);
return ::_releasedir(di);
return local::releasedir(di);
}
}
}

1
src/removexattr.cpp

@ -29,7 +29,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

2
src/rename.cpp

@ -283,6 +283,8 @@ namespace mergerfs
const ugid::Set ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.branches_lock);
config.open_cache.erase(oldpath);
if(config.create->path_preserving() && !config.ignorepponrename)
return _rename_preserve_path(config.getattr,
config.rename,

1
src/rmdir.cpp

@ -30,7 +30,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

11
src/setxattr.cpp

@ -37,7 +37,6 @@ static const char SECURITY_CAPABILITY[] = "security.capability";
using std::string;
using std::vector;
using mergerfs::Policy;
using mergerfs::FuseFunc;
using namespace mergerfs;
@ -229,6 +228,9 @@ _setxattr_controlfile_func_policy(Config &config,
if((flags & XATTR_CREATE) == XATTR_CREATE)
return -EEXIST;
if(funcname == "open")
config.open_cache.clear();
rv = config.set_func_policy(funcname,attrval);
if(rv == -1)
return -errno;
@ -248,6 +250,9 @@ _setxattr_controlfile_category_policy(Config &config,
if((flags & XATTR_CREATE) == XATTR_CREATE)
return -EEXIST;
if(categoryname == "search")
config.open_cache.clear();
rv = config.set_category_policy(categoryname,attrval);
if(rv == -1)
return -errno;
@ -340,6 +345,10 @@ _setxattr_controlfile(Config &config,
attr[3],
attrval,
flags);
else if((attr[2] == "cache") && (attr[3] == "open"))
return _setxattr_uint64_t(attrval,
flags,
config.open_cache.timeout);
break;
default:

1
src/truncate.cpp

@ -32,7 +32,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

76
src/unlink.cpp

@ -1,5 +1,5 @@
/*
Copyright (c) 2016, Antonio SJ Musumeci <trapexit@spawn.link>
Copyright (c) 2019, Antonio SJ Musumeci <trapexit@spawn.link>
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
@ -14,13 +14,6 @@
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
#include <fuse.h>
#include <unistd.h>
#include <string>
#include <vector>
#include "config.hpp"
#include "errno.hpp"
#include "fs_base_unlink.hpp"
@ -29,57 +22,66 @@
#include "rwlock.hpp"
#include "ugid.hpp"
#include <fuse.h>
#include <string>
#include <vector>
#include <unistd.h>
using std::string;
using std::vector;
using mergerfs::Policy;
static
int
_unlink_loop_core(const string *basepath,
const char *fusepath,
const int error)
namespace local
{
static
int
unlink_loop_core(const string *basepath_,
const char *fusepath_,
const int error_)
{
int rv;
string fullpath;
fs::path::make(basepath,fusepath,fullpath);
fs::path::make(basepath_,fusepath_,fullpath);
rv = fs::unlink(fullpath);
return error::calc(rv,error,errno);
}
return error::calc(rv,error_,errno);
}
static
int
_unlink_loop(const vector<const string*> &basepaths,
const char *fusepath)
{
static
int
unlink_loop(const vector<const string*> &basepaths_,
const char *fusepath_)
{
int error;
error = -1;
for(size_t i = 0, ei = basepaths.size(); i != ei; i++)
for(size_t i = 0, ei = basepaths_.size(); i != ei; i++)
{
error = _unlink_loop_core(basepaths[i],fusepath,error);
error = local::unlink_loop_core(basepaths_[i],fusepath_,error);
}
return -error;
}
}
static
int
_unlink(Policy::Func::Action actionFunc,
static
int
unlink(Policy::Func::Action actionFunc_,
const Branches &branches_,
const uint64_t minfreespace,
const char *fusepath)
{
const uint64_t minfreespace_,
const char *fusepath_)
{
int rv;
vector<const string*> basepaths;
rv = actionFunc(branches_,fusepath,minfreespace,basepaths);
rv = actionFunc_(branches_,fusepath_,minfreespace_,basepaths);
if(rv == -1)
return -errno;
return _unlink_loop(basepaths,fusepath);
return local::unlink_loop(basepaths,fusepath_);
}
}
namespace mergerfs
@ -87,17 +89,19 @@ namespace mergerfs
namespace fuse
{
int
unlink(const char *fusepath)
unlink(const char *fusepath_)
{
const fuse_context *fc = fuse_get_context();
const Config &config = Config::get(fc);
const ugid::Set ugid(fc->uid,fc->gid);
const rwlock::ReadGuard readlock(&config.branches_lock);
return _unlink(config.unlink,
config.open_cache.erase(fusepath_);
return local::unlink(config.unlink,
config.branches,
config.minfreespace,
fusepath);
fusepath_);
}
}
}

1
src/utimens.cpp

@ -31,7 +31,6 @@
using std::string;
using std::vector;
using mergerfs::Policy;
static
int

Loading…
Cancel
Save