NAME
Data::RecordStore - Simple store for text and byte dataSYNPOSIS
use Data::RecordStore;
$store = Data::RecordStore->open_store( $directory );
$data = "TEXT OR BYTES";
# the first record id is 1
my $id = $store->stow( $data );
my $new_or_recycled_id = $store->next_id;
$store->stow( $new_data, $new_or_recycled_id );
my $val = $store->fetch( $some_id );
my $count = $store->entry_count;
# delete the old record, make its id available for future
# records
$store->recycle_id( $id_to_recycle );
$store->delete_record( $id_to_remove ); #deletes the old record
my $has_id = $store->has_id( $someother_id );
$store->empty; # clears out store completelyDESCRIPTION
Data::RecordStore is a simple way to store serialized text or byte data. It is written entirely in perl with no non-core dependencies. It is designed to be both easy to set up and easy to use. Space is automatically reclaimed when records are reycled or deleted.
Transactions (see below) can be created that stow and recycle records. They come with the standard commit and rollback methods. If a process dies in the middle of a commit or rollback, the operation can be reattempted. Incomplete transactions are obtained by the store's 'list_transactions' method.
Data::RecordStore operates directly and instantly on the file system. It is not a daemon or server and is not thread safe. It can be used in a thread safe manner if the controlling program uses locking mechanisms.
METHODS
open_store( options )
Constructs a data store according to the options.
Options
- BASE_PATH
-
The directory to construct this record store in.
- MODE
-
'MULTI' or 'SINGLE'. If MULTI, flocking will be used for any writes to the main index in order to coordinate multiple processes updating the store.
sync()
Syncs the data store to persistance. Stub implementation here. Subclasses, notably Data::RecordStore::Redis may cache stows and deletes and this method will sync those actions to persistance.
pending()
Returns the approximate number of bytes needing to be written.
size()
Reports the size in bytes of the items in the record store. Remember that the minimum size of an item will be the block size 4096. This size does not include the index.
provider_id()
Returns a value that identifies this record store uniquely.
detect_version( directory )
Tries to detect version of the record store in the directory. Returns undef if it is unable to detect it.
start_transaction()
Creates and returns a transaction object
use_transaction()
Returns the current transaction. If there is no current transaction, it creates one and returns it.
commit_transaction()
Commits the current transaction, if any.
rollback_transaction()
Rolls back the current transaction, if any.
list_transactions
Returns a list of currently existing transaction objects not marked TRA_DONE.
last_updated( id )
Returns the timestamp that this record was last written to the database.
stow( data, optionalID )
This saves the text or byte data to the record store. If an id is passed in, this saves the data to the record for that id, overwriting what was there. If an id is not passed in, it creates a new record store.
Returns the id of the record written to.
fetch( id )
Returns the record associated with the ID. If the ID has no record associated with it, undef is returned.
active_entry_count
Scans the index and returns a count of how many records are marked as having active data
record_count
Scans the silos and returns a count of how many records are contained in them.
entry_count
Returns how many record ids exist. This is equal to the highest ID that has been assigned minus the number of pending recycles. It is different from the record count, as entries may be marked deleted.
record_count
Return how many records there actually are
delete_record( id )
Removes the entry with the given id from the store, freeing up its space. It does not reuse the id.
has_id( id )
Returns true if an record with this id exists in the record store.next_id
This sets up a new empty record and returns the id for it.
empty()
This empties out the entire record store completely. Use only if you mean it.
empty_recycler()
Clears out all data from the recycler.recycle( id, keep_data_flag )
Ads the id to the recycler, so it will be returned when next_id is called.
This removes the data occupied by the id, freeing up space unles
keep_data_flag is set to true.lock( @names )
Adds an advisory (flock) lock for each of the unique names given. This may not be called twice in a row without an unlock in between.
unlock
Unlocks all names locked by this thread
HELPER PACKAGE
Data::RecordStore::Transaction
HELPER DESCRIPTION
A transaction that can collect actions on the record store and then writes them as a block.
HELPER SYNOPSIS
my $transaction = $store->create_transaction;
print join(",", $transaction->get_update_time, $transaction->get_process_id, $transaction->get_state, $transaction->get_id );
my $new_id = $transaction->stow( $data );
my $new_or_recycled_id = $store->next_id;
$transaction->stow( "MORE DATA", $new_or_recycled_id );
$transaction->delete_record( $someid ); $transaction->recycle_id( $dead_id );
if( $is_good ) { $transaction->commit; } else { $transaction->rollback; }
# # Get a list of transactions that are old and probably stale. # for my $trans ($store->list_transactions) {
next if $trans->get_udpate_time > $too_old;
my $state = $trans->get_state;
if( $state == Data::RecordStore::Transaction::TRA_IN_COMMIT
|| $state == Data::RecordStore::Transaction::TRA_CLEANUP_COMMIT )
{
$trans->commit;
}
elsif( $state == Data::RecordStore::Transaction::TRA_IN_ROLLBACK
|| $state == Data::RecordStore::Transaction::TRA_CLEANUP_ROLLBACK )
{
$trans->rollback;
}
elsif( $state == Data::RecordStore::Transaction::TRA_ACTIVE )
{
# commit or rollback, depending on preference
}
}HELPER METHODS
get_update_time
Returns the epoch time when the last time this was updated.
get_process_id
Returns the process id that last wrote to this transaction.
get_state
Returns the state of this process. Values are TRA_ACTIVE TRA_IN_COMMIT TRA_IN_ROLLBACK TRA_COMMIT_CLEANUP TRA_ROLLBACK_CLEANUP TRA_DONE
get_id
Returns the ID for this transaction, which is the same as its position in the transaction index plus one.
stow( $data, $optional_id )
Stores the data given. Returns the id that the data was stowed under. If the id is not given, this generates one from the record store. The data stored this way is really stored in the record store, but the index is not updated until a commit happens. That means it is not reachable from the store until the commit.
delete_record( $id )
Marks that the record associated with the id is to be deleted when the transaction commits.
recycle_id( $id )
Marks that the record associated with the id is to be deleted and its id recycled when the transaction commits.
commit()
Commit applies
unlink_store
Removes the file for this record store entirely from the file system.
AUTHOR Eric Wolf coyocanid@gmail.com
COPYRIGHT AND LICENSE
Copyright (c) 2015-2018 Eric Wolf. All rights reserved.
This program is free software; you can redistribute it
and/or modify it under the same terms as Perl itself.