NAME

File::FStore - Module for interacting with file stores

VERSION

version v0.04

SYNOPSIS

use File::FStore;

my File::FStore $store = File::FStore->new(path => '...');

my @files = $store->query(properties => mediasubtype => 'image/png', order => 'asc', offset => 10, limit => 5);

my $fh = $store->query(digests => 'sha-3-224' => '6b4e03423667dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7')->open;

This package provides access to a hash based/final state file store in Fellig format. Files in such a store are considered final, meaning their content will no longer be altered.

In addition to digests (hashes) some file level metadata is kept in the store (see "PROPERTIES" in File::FStore::File). This metadata is mainly used to ensure the integrity of the store. Other metadata is intentionally not supported. For storage of other metadata see Data::TagDB and "db". For reading metadata and file analysis see File::Information and "fii".

METHODS

create

my File::FStore $store = File::FStore->create(path => ..., ...);

Creates a new file store. dies if the store cannot be created or already exists or invalid options are passed. Takes the same options as "new" plus the following:

digests

List of digests to be used in the store. Each digest is given in the universal tag format (or utag) (e.g. sha-3-224. The list can be passed as a arrayref or as a comma seperated list.

The list can contain digests that are not supported by the system this runs on. They may for example still be used with import/export functions.

The list can be adjusted at a later time.

new

my File::FStore $store = File::FStore->new(path => ..., ...);

Creates a new instance of the store and opens it.

The following options are supported:

db: A Data::TagDB object. See "db".
extractor: A Data::URIID object. See "extractor".
fii: A File::Information object. See "fii".
path: The path to the store.
weak: Marks the value for db, extractor, fii as weak. If only a specific one needs needs to be weaken use "attach".

close

$store->close;

Closes the store. Any interaction with this object or any related objects after this call is invalid.

in_transaction

$db->in_transaction(ro => sub { ....});
# or:
$db->in_transaction(rw => sub { ....});

Runs a block of code (a subref) inside a transaction.

The passed block is run in a transaction. The transaction is commited after the code finishes.

The type of the transaction can be ro (read only) or rw (read-write). The module may optimise based on this information. If a write operation is performed in a transaction that is marked ro the behaviour is unspecified.

Calls to this method can be stacked freely. For example the following is valid:

$store->in_transaction(ro => sub {
    # do some read...
    $store->in_transaction(rw => sub {
        # do some write...
    });
    # do more reading, writing is invalid here
});

Note: If the code dies the transaction is aborted and the error is raised again. Note that this affects all currently open transactions (as per stacking). If the (parent) transaction is already aborted when this method is called the code block might not be run at all.

Note: Data written might only be visible to other handles of the same database once all transactions have been finished.

Note: It is undefined what the state of @_ is within the callback.

query

my File::FStore::File $file = $store->query(...);
# or:
my @files = $store->query(...);

# e.g.:
my File::FStore::File $file = $store->query(digests => 'sha-3-512' => $digest);
# or:
my File::FStore::File $file = $store->query(digests => {'sha-3-512' => $digest});
# or:
foreach my File::FStore::File $file ($store->query(properties => size => 64)) {
    # ...
}

Returns files matching the given query. If called in scalar context the method will return the file if the query matches one file. If it matches zero or more than one it will die.

The query contains of commands followed by arguments. It is optimised internally for performance, so order of arguments does not matter.

Currently the following commands are defined:

properties

This takes a key and value or a single hashref with properties to match exactly. The value can also be an arrayref of values (which are or-ed).

digests

This works the same as properties but matches the digests.

all

Selects all files. Takes no arguments. This should be used with care as a very long list might be returned.

dbname

Selects files by their dbname (see "dbname" in File::FStore::File). Takes a single filename or a list of filenames (as an arrayref).

ise

Selects files by their ise (such as their "contentise" in File::FStore::File, or "inodeise" in File::FStore::File). Takes a single ISE, Data::Identifier, Data::TagDB::Tag, Data::URIID::Base, or a list of them (as an arrayref) or a Data::Identifier::Cloudlet.

If a Data::Identifier::Cloudlet all entries are used (see "entries" in Data::Identifier::Cloudlet), not just root entries.

min_size

Selects files by their minimum size. This uses the size value of the properties domain.

max_size

Selects files by their maximum size. This uses the size value of the properties domain.

limit

Limits the number of returned files. It is undefined what happens if this is used in scalar context.

offset

The offset into the list. Often used with limit to implement pagination. Note: It is important to also give order when using this, as otherwise the order is not stable.

order

The order in which to return the files. One of asc or desc.

scrub

$store->scrub;

Cleans up the store, including removing dangling links.

This is normally only needed if files where removed or files got corrupted, or store settings have been altered. However it is a good idea to call this method once in a while to maintain the store over a long time.

It is typical application to call "scan" after this call to check and regenerate active symlinks.

Note: This method may take some time to run. Generally speaking the call takes more time the larger the store is.

scan

$store->scan;
# or:
$store->scan(%opts);

Scans the store for existing and new files. As this method checks all files in the store and may do calculations this method might run several seconds.

The following options are supported:

update: One of all (default), new, or none. This determines whether "update" in File::FStore::File is called on all files, only new ones, or none.
no_digests: Passed to "update" in File::FStore::File.
on_pre_set: See "update" in File::FStore::File. Also called on each file if no update is performed.
on_post_set: See "update" in File::FStore::File. Also called on each file if no update is performed.

fix

$store->fix(qw(fixes...));

Runs maintenance/fixes to the store.

Takes the names of the fixes to run as arguments. The order does not matter, as the method reorders them internally.

On any error this method dies.

Currently the following fixes are supported:

remove-inode: Removes the inode properties from the store. This is useful when transfering the store to another filesystem, or if the store is managed externally (e.g. via an vcs).
remove-inodeise: Removes the inodeise properties from the store. This is useful when transfering the store to another filesystem, or if the store is managed externally (e.g. via an vcs). In contrast to remove-inode this might not be needed, depending on which schema for inodeise was used.
remove-mediasubtype: Removes the mediasubtype properties from the store. This is useful when you imported invalid mediasubtypes from a bad source. Adding scan to the list of fixes will add correct values back.
scan: Runs "scan" with default options.
scrub: Runs "scrub".
upgrade: Runs "upgrade" in File::FStore::Migration.

new_adder

my File::FStore::Adder $adder = $store->new_adder;

Create a new File::FStore::Adder.

migration

my File::FStore::Migration $migration = $self->migration;

Returns a new File::FStore::Migration object for this store.

export

$store->export($handle, %opts);

Exports the store metadata.

The following (all optional) options are supported:

format: The format to use. Currently supported is json for the classic JSON format. And valuefile for universal tag based ValueFile output.
list: A arrayref to a list of files to include.
query: A arrayref with a query in the same format as "query" takes it.

Note: If you want a stringified result you can use a memory handle as documented in "open" in perlfunc. E.g.: open(my $fh, '>', \$result).

import_data

$store->import_data($handle);
# or:
$store->import_data($handle, format => ...);

This method allows importing data into the database from an open handle.

The data is imported the same way as per "set" in File::FStore::File including all safety checks.

The following (all optional) options are supported:

format: The format to use. Currently supported is json for the classic JSON format.

attach

$self->attach(key => $obj, ...);
# or:
$self->attach(key => $obj, ..., weak => 1);

Attaches objects of the given type. Takes the same list of objects as "new".

If an object is allready attached for the given key this method dies unless the object is actually the same.

If weak is set to a true value the object reference becomes weak.

db

my Data::TagDB $db = $store->db;
# or:
my Data::TagDB $db = $store->db(default => $def);

Returns the instance of Data::TagDB if any was given via "new".

If no value is known returns the value of the option default (if passed) or dies.

extractor

my Data::URIID $extractor = $store->extractor;
# or:
my Data::URIID $extractor = $store->extractor(default => $def);

Returns the instance of Data::URIID if any was given via "new".

If no value is known returns the value of the option default (if passed) or dies.

fii

my File::Information $fii = $store->fii;

Returns the instance of File::Information passed via "new" or a internally created one if none were given.

Note: If the option default is passed, it is ignored with no error.

AUTHOR

Löwenfelsen UG (haftungsbeschränkt) <support@loewenfelsen.net>

COPYRIGHT AND LICENSE

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)

To install File::FStore, copy and paste the appropriate command in to your terminal.

cpanm

cpanm File::FStore

CPAN shell

perl -MCPAN -e shell
install File::FStore

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)