NAME
Time::Progress::Stored - Report progress + store and retrieve the current status
DESCRIPTION
This module helps if you have a long running process which reports progress, but you need to actually display the progress to the user in a different process.
Typically this is a long running web request in the web server or in a job queue qorker, while the web browser periodically sends ajax requests to get updated on the current progress status to show in a progress bar.
Time::Progress::Stored stores the progress report as the worker performs its job, and retrieves it from elsewhere where the report is needed.
The report is a hashref with the following details:
| id | "progress-1454518121172" |
| max | 262 |
| current | 2 |
| activity | "Importing Business Units" |
| elapsed_seconds | "1" |
| elapsed_time | " 0:01" |
| finish_time | "Wed Feb 3 16:50:54 2016" |
| percent | 0.8 |
| percent_string | " 0.8%" |
| remaining_seconds | "130" |
| remaining_time | " 2:10" |
| is_done | 0 |SYNOPSIS
### In the worker process
my $progess_id = ...; # Probably comes from the client
my $max = @items * 2; # Let's say two passes over @items
my $redis = Redis->new( ... );
my $progress = Time::Progress::Stored->new({
max => $max,
progress_id => $progress_id,
storage => Progress::Stored::Storage::Redis->new({ redis => $redis }),
});
# Do the work
for my $item (@items) {
# Do stuff
$progress->advance("Doing the first thing");
}
for my $item (@items) {
# Do other stuff
$progress->advance("Doing other stuff with $item->{name}");
}
$progress->done();
### HTTP endpoint, e.g. a Mojo controller for /progress/:progress_id
sub get {
my $self = shift;
my $progress_id = $self->param("progress_id");
my $storage = Time::Progress::Stored::Storage::Redis->new({
redis => $self->redis,
});
my $response_json = $storage->retrieve($progress_id)
or return $self->render(status => 404, json => {});
return $self->render(json => { progress => $response_json });
}Synopsis breakdown
Worker
As the long running piece of work comes in, start the progress reporting.
Create the $progress object with a storage to store the progress reports. Initialize it with the number of actions you want to perform.
For each time something is done, call $progress->advance(). This might update the actual report.
By default only about 100 steps will be reported (each percent progressed). You can increase this by specifying the attribute "report_every", e.g. to report every single time something happens, set it to 1 (this might slow things down though).
"advance" can optionally be called with an activity description string, indicating what's happening right now. This can be large scale steps, or include details about this specific action being performed.
At the end, call $progress->done, just to make sure the reporting knows it has reached the end. This will set is_done: 1 in the report.
HTTP endpoint
The client should send requests to the endpoint periodically to get the current progress report. The request contains the progress id, and it's simply fetched and returned as a JSON payload.
See above for the contents of the payload.
Client with long running web request
This is for when the long running process is running as one web request.
A web page would typically provide a progress id while kickking off the long running process.
The client then sends ajax requests to the HTTP endpoint (using the id) every second to get updates on the progress. It does this until the long running progress has responded.
If you look at the report payload you'll see there is a variety of information to provide user feedback, e.g. percent done, time left, what's going on etc.
Client with asynchronous worker process
This is for when the long running process is running as a background process, e.g. in a job queue.
A web page would typically kick off the long running process. Instead of the client passing in a progress id, the $progress object defaults to a unique id (or it can just make one up), and the HTTP endpoint can report $progress->progress_id back to the client.
The client then sends ajax requests to the HTTP endpoint (using the id) every second to get updates on the progress. It does this until the progress report "is_done" is set to 1.
Storage backends
Time::Progress::Stored can use various backends to store the status.
Current backends are Redis and Memory. They are tiny and it would be simple to write another backend for e.g. a file, or using one of the Cache or CHI modules. Patches more than welcome.
Pitfalls
If you're testing with a single threaded test server like Morbo, the web browser's Ajax requests won't be processed until the long running process has finished, so you'll never see the progress until it's all done.
ATTRIBUTES
max
The number of times you're planning to call ->advance().
storage
A Storage object to store reports in.
Default: Time::Progress::Stored::Storage::Memory, but that's probably only useful for testing.
progress_id
The id to identify this progress report with. Either set by the client, or defaulted to a unique URL-param safe value you can return to the client.
report_every
Number of calls to ->advance() for every time a new progress report is generated and stored.
Default: a value that results in a report for every percent of progress.
current
The current iteration. You could in theory set this yourself, but it's recommended to call ->advance().
current_activity
Same as ->current, but for the current activity.
METHODS
advance($activity?)
Advance the progress one iteration. Optionally set the current $activity.
done()
Mark the progress as being completed, and set the is_done key in the report.
DEVELOPMENT
Author
Johan Lindstrom, <johanl [AT] cpan.org>
Contributors
Source code
https://github.com/jplindstrom/p5-Time-Progress-Stored
Bug reports
Please report any bugs or feature requests on GitHub:
https://github.com/jplindstrom/p5-Time-Progress-Stored/issues.
Caveats
COPYRIGHT & LICENSE
Copyright 2016- Broadbean Technologies, All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
ACKNOWLEDGEMENTS
Thanks to Broadbean for providing time to open source this during one of the regular Hack-days.
This module uses the excellent Time::Progress module, which you should be using if you just need a progress bar in a command line script.