type BTreeIndex struct {
sync.RWMutex
LessFunction
*btree.BTree
}
BTreeIndex is an implementation of the Index interface using google/btree.
func (i *BTreeIndex) Delete(key string)
Delete removes the given key (only) from the BTree tree.
func (i *BTreeIndex) Initialize(less LessFunction, keys <-chan string)
Initialize populates the BTree tree with data from the keys channel, according to the passed less function. It's destructive to the BTreeIndex.
func (i *BTreeIndex) Insert(key string)
Insert inserts the given key (only) into the BTree tree.
func (i *BTreeIndex) Keys(from string, n int) []string
Keys yields a maximum of n keys in order. If the passed 'from' key is empty, Keys will return the first n keys. If the passed 'from' key is non-empty, the first key in the returned slice will be the key that immediately follows the passed key, in key order.
type Compression interface {
Writer(dst io.Writer) (io.WriteCloser, error)
Reader(src io.Reader) (io.ReadCloser, error)
}
Compression is an interface that Diskv uses to implement compression of data. Writer takes a destination io.Writer and returns a WriteCloser that compresses all data written through it. Reader takes a source io.Reader and returns a ReadCloser that decompresses all data read through it. You may define these methods on your own type, or use one of the NewCompression helpers.
func NewGzipCompression() Compression
NewGzipCompression returns a Gzip-based Compression.
func NewGzipCompressionLevel(level int) Compression
NewGzipCompressionLevel returns a Gzip-based Compression with the given level.
func NewZlibCompression() Compression
NewZlibCompression returns a Zlib-based Compression.
func NewZlibCompressionLevel(level int) Compression
NewZlibCompressionLevel returns a Zlib-based Compression with the given level.
func NewZlibCompressionLevelDict(level int, dict []byte) Compression
NewZlibCompressionLevelDict returns a Zlib-based Compression with the given level, based on the given dictionary.
type Diskv struct {
Options
// contains filtered or unexported fields
}
Diskv implements the Diskv interface. You shouldn't construct Diskv structures directly; instead, use the New constructor.
func New(o Options) *Diskv
New returns an initialized Diskv structure, ready to use. If the path identified by baseDir already contains data, it will be accessible, but not yet cached.
func (d *Diskv) Erase(key string) error
Erase synchronously erases the given key from the disk and the cache.
func (d *Diskv) EraseAll() error
EraseAll will delete all of the data from the store, both in the cache and on the disk. Note that EraseAll doesn't distinguish diskv-related data from non- diskv-related data. Care should be taken to always specify a diskv base directory that is exclusively for diskv data.
func (d *Diskv) Has(key string) bool
Has returns true if the given key exists.
func (d *Diskv) Import(srcFilename, dstKey string, move bool) (err error)
Import imports the source file into diskv under the destination key. If the destination key already exists, it's overwritten. If move is true, the source file is removed after a successful import.
func (d *Diskv) Keys(cancel <-chan struct{}) <-chan string
Keys returns a channel that will yield every key accessible by the store, in undefined order. If a cancel channel is provided, closing it will terminate and close the keys channel.
func (d *Diskv) KeysPrefix(prefix string, cancel <-chan struct{}) <-chan string
KeysPrefix returns a channel that will yield every key accessible by the store with the given prefix, in undefined order. If a cancel channel is provided, closing it will terminate and close the keys channel. If the provided prefix is the empty string, all keys will be yielded.
func (d *Diskv) Read(key string) ([]byte, error)
Read reads the key and returns the value. If the key is available in the cache, Read won't touch the disk. If the key is not in the cache, Read will have the side-effect of lazily caching the value.
func (d *Diskv) ReadStream(key string, direct bool) (io.ReadCloser, error)
ReadStream reads the key and returns the value (data) as an io.ReadCloser. If the value is cached from a previous read, and direct is false, ReadStream will use the cached value. Otherwise, it will return a handle to the file on disk, and cache the data on read.
If direct is true, ReadStream will lazily delete any cached value for the key, and return a direct handle to the file on disk.
If compression is enabled, ReadStream taps into the io.Reader stream prior to decompression, and caches the compressed data.
func (d *Diskv) Write(key string, val []byte) error
Write synchronously writes the key-value pair to disk, making it immediately available for reads. Write relies on the filesystem to perform an eventual sync to physical media. If you need stronger guarantees, see WriteStream.
func (d *Diskv) WriteStream(key string, r io.Reader, sync bool) error
WriteStream writes the data represented by the io.Reader to the disk, under the provided key. If sync is true, WriteStream performs an explicit sync on the file as soon as it's written.
bytes.Buffer provides io.Reader semantics for basic data types.
type Index interface {
Initialize(less LessFunction, keys <-chan string)
Insert(key string)
Delete(key string)
Keys(from string, n int) []string
}
Index is a generic interface for things that can provide an ordered list of keys.
type LessFunction func(string, string) bool
LessFunction is used to initialize an Index of keys in a specific order.
type Options struct {
BasePath string
Transform TransformFunction
CacheSizeMax uint64 // bytes
PathPerm os.FileMode
FilePerm os.FileMode
Index Index
IndexLess LessFunction
Compression Compression
}
Options define a set of properties that dictate Diskv behavior. All values are optional.
type TransformFunction func(s string) []string
TransformFunction transforms a key into a slice of strings, with each element in the slice representing a directory in the file path where the key's entry will eventually be stored.
For example, if TransformFunc transforms "abcdef" to ["ab", "cde", "f"], the final location of the data file will be <basedir>/ab/cde/f/abcdef
| Name | Synopsis |
|---|---|
| .. | |
| examples | |
| content-addressable-store | |
| super-simple-store | |