NAME
File::FStore - Module for interacting with file stores
VERSION
version v0.02
SYNOPSIS
use File::FStore;
my File::FStore $store = File::FStore->new(path => '...');
my @files = $store->query(properties => mediasubtype => 'image/png', order => 'asc', offset => 10, limit => 5);
my $fh = $store->query(digests => 'sha-3-224' => '6b4e03423667dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7')->open;
This package provides access to a hash based/final state file store in Fellig format. Files in such a store are considered final, meaning their content will no longer be altered.
In addition to digests (hashes) some file level metadata is kept in the store (see "PROPERTIES" in File::FStore::File). This metadata is mainly used to ensure the integrity of the store. Other metadata is intentionally not supported. For storage of other metadata see Data::TagDB and "db". For reading metadata and file analysis see File::Information and "fii".
METHODS
create
my File::FStore $store = File::FStore->create(path => ..., ...);
Creates a new file store. die
s if the store cannot be created or already exists or invalid options are passed. Takes the same options as "new" plus the following:
digests
-
List of digests to be used in the store. Each digest is given in the universal tag format (or utag) (e.g.
sha-3-224
. The list can be passed as a arrayref or as a comma seperated list.The list can contain digests that are not supported by the system this runs on. They may for example still be used with import/export functions.
The list can be adjusted at a later time.
new
my File::FStore $store = File::FStore->new(path => ..., ...);
Creates a new instance of the store and opens it.
The following options are supported:
db
-
A Data::TagDB object. See "db".
extractor
-
A Data::URIID object. See "extractor".
fii
-
A File::Information object. See "fii".
path
-
The path to the store.
close
$store->close;
Closes the store. Any interaction with this object or any related objects after this call is invalid.
in_transaction
$db->in_transaction(ro => sub { ....});
# or:
$db->in_transaction(rw => sub { ....});
Runs a block of code (a subref) inside a transaction.
The passed block is run in a transaction. The transaction is commited after the code finishes.
The type of the transaction can be ro
(read only) or rw
(read-write). The module may optimise based on this information. If a write operation is performed in a transaction that is marked ro
the behaviour is unspecified.
Calls to this method can be stacked freely. For example the following is valid:
$store->in_transaction(ro => sub {
# do some read...
$store->in_transaction(rw => sub {
# do some write...
});
# do more reading, writing is invalid here
});
Note: If the code die
s the transaction is aborted and the error is raised again. Note that this affects all currently open transactions (as per stacking). If the (parent) transaction is already aborted when this method is called the code block might not be run at all.
Note: Data written might only be visible to other handles of the same database once all transactions have been finished.
Note: It is undefined what the state of @_
is within the callback.
query
my File::FStore::File $file = $store->query(...);
# or:
my @files = $store->query(...);
# e.g.:
my File::FStore::File $file = $store->query(digests => 'sha-3-512' => $digest);
# or:
my File::FStore::File $file = $store->query(digests => {'sha-3-512' => $digest});
# or:
foreach my File::FStore::File $file ($store->query(properties => size => 64)) {
# ...
}
Returns files matching the given query. If called in scalar context the method will return the file if the query matches one file. If it matches zero or more than one it will die
.
The query contains of commands followed by arguments. It is optimised internally for performance, so order of arguments does not matter.
Currently the following commands are defined:
properties
-
This takes a key and value or a single hashref with properties to match exactly.
digests
-
This works the same as
properties
but matches the digests. all
-
Selects all files. Takes no arguments. This should be used with care as a very long list might be returned.
dbname
-
Selects files by their dbname (see "dbname" in File::FStore::File). Takes a single filename or a list of filenames (as an arrayref).
limit
-
Limits the number of returned files. It is undefined what happens if this is used in scalar context.
offset
-
The offset into the list. Often used with
limit
to implement pagination. Note: It is important to also giveorder
when using this, as otherwise the order is not stable. order
-
The order in which to return the files. One of
asc
ordesc
.
scrub
$store->scrub;
Cleans up the store, including removing dangling links.
This is normally only needed if files where removed or files got corrupted, or store settings have been altered. However it is a good idea to call this method once in a while to maintain the store over a long time.
It is typical application to call "scan" after this call to check and regenerate active symlinks.
Note: This method may take some time to run. Generally speaking the call takes more time the larger the store is.
scan
$store->scan;
# or:
$store->scan(%opts);
Scans the store for existing and new files. As this method checks all files in the store and may do calculations this method might run several seconds.
The following options are supported:
update
-
One of
all
(default),new
, ornone
. This determines whether "update" in File::FStore::File is called on all files, only new ones, or none. no_digests
-
Passed to "update" in File::FStore::File.
fix
$store->fix(qw(fixes...));
Runs maintenance/fixes to the store.
Takes the names of the fixes to run as arguments. The order does not matter, as the method reorders them internally.
On any error this method die
s.
Currently the following fixes are supported:
remove-inode
-
Removes the inode properties from the store. This is useful when transfering the store to another filesystem, or if the store is managed externally (e.g. via an vcs).
remove-mediasubtype
-
Removes the mediasubtype properties from the store. This is useful when you imported invalid mediasubtypes from a bad source. Adding
scan
to the list of fixes will add correct values back. scan
-
Runs "scan" with default options.
scrub
-
Runs "scrub".
new_adder
my File::FStore::Adder $adder = $store->new_adder;
Create a new File::FStore::Adder.
export
$store->export($handle, %opts);
Exports the store metadata.
The following (all optional) options are supported:
format
-
The format to use. Currently supported is
json
for the classic JSON format. Andvaluefile
for universal tag based ValueFile output. list
-
A arrayref to a list of files to include.
query
-
A arrayref with a query in the same format as "query" takes it.
Note: If you want a stringified result you can use a memory handle as documented in "open" in perlfunc. E.g.: open(my $fh, '>', \$result)
.
import_data
$store->import_data($handle);
# or:
$store->import_data($handle, format => ...);
This method allows importing data into the database from an open handle.
The data is imported the same way as per "set" in File::FStore::File including all safety checks.
The following (all optional) options are supported:
format
-
The format to use. Currently supported is
json
for the classic JSON format.
db
my Data::TagDB $db = $store->db;
# or:
my Data::TagDB $db = $store->db(default => $def);
Returns the instance of Data::TagDB if any was given via "new".
If no value is known returns the value of the option default
(if passed) or die
s.
extractor
my Data::URIID $extractor = $store->extractor;
# or:
my Data::URIID $extractor = $store->extractor(default => $def);
Returns the instance of Data::URIID if any was given via "new".
If no value is known returns the value of the option default
(if passed) or die
s.
fii
my File::Information $fii = $store->fii;
Returns the instance of File::Information passed via "new" or a internally created one if none were given.
Note: If the option default
is passed, it is ignored with no error.
AUTHOR
Löwenfelsen UG (haftungsbeschränkt) <support@loewenfelsen.net>
COPYRIGHT AND LICENSE
This software is Copyright (c) 2025 by Löwenfelsen UG (haftungsbeschränkt) <support@loewenfelsen.net>.
This is free software, licensed under:
The Artistic License 2.0 (GPL Compatible)