NAME
File::BufferedInput - Large and giant text file performance buffered reader.
SYNOPSIS
use File::BufferedInput;
# create new object with default options, buffer size 20MB and ANSI text file
# file argument can be file name or file handle
my $fileobj = File::BufferedInput->new(file=>$filename);
# or create new object with custom options, set buffer size 50MB and UTF-8 text file
my $fileobj = File::BufferedInput->new(file=>$filename, buffer=>50_000_000, utf8=>1);
# or create new object then set the file name/handle
my $fileobj = File::BufferedInput->new();
$fileobj->file($filename_or_handle);
# now loop through all the text file lines straight forward
my $counter = 0;
while (my $line = $fileobj->line()) {#loop through the file lines sequentially
$counter++;
# print "$counter)- $line\n";
}
# start again from the begaining of the file
#$fileobj->rewind()
$fileobj->close(); # close the file and frees the memory used for the block
print "$counter lines found\n";
DESCRIPTION
This module solves the problem with reading large and huge text files in Perl. It is designed to read only block by block as needed. It does not load the whole file into memory, it only reads one block at a time and once the last sequential line reached, it reads the next block from the file and frees the previous block from memory, so at all times only one block of the file is kept in menory.
For example if you are reading a 2GB file, once you start reading lines from the file, the module reads the first block from the file on disk, while you loop through the lines, when you reach the line at the end of the read block, the module delete this block from memory and read the next block from the file on disk and parses it to lines and so on.
SEE ALSO
AUTHOR
Ahmed Amin Elsheshtawy, <support@mewsoft.com> Website: http://www.mewsoft.com
COPYRIGHT AND LICENSE
Copyright (C) 2014 by Ahmed Amin Elsheshtawy support@mewsoft.com http://www.mewsoft.com
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.