NAME

RPC::ExtDirect::Intro - A gentle(ish) introduction to RPC::ExtDirect

DESCRIPTION

What is Ext.Direct?

Ext.Direct is a high level RPC-over-HTTP protocol provided out of the box with Ext JS and Sencha Touch JavaScript frameworks. It is server agnostic, provided that the server side stack is conformant to the Ext.Direct specification.

RPC::ExtDirect is a fully featured implementation of Ext.Direct server stack in Perl, compatible with Ext JS 4.x and 5.x, and Sencha Touch 2.x.

What Ext.Direct is for?

The primary goal for Ext.Direct is easy and fast integration of server components with HTML5 applications. Client side stack is built in frameworks' cores and is used by many components like data Stores, Forms, Grids, Charts, etc. Ext.Direct supports request batching, file uploads, event polling and many other features.

Besides simplicity and ease of use, Ext.Direct allows to achieve very clean code and issue separation both on server and client sides, which in turn results in simplified code, greater overall software quality and shorter development times.

From Perl module developer perspective, Ext.Direct is just a transport layer; it doesn't matter if a method is called from Perl code or through Ext.Direct. This approach, in particular, allows for multi-tiered testing:

Server side methods can be tested without setting up HTTP environment with the usual tools like Test::More
Server side classes can be tested as a whole via Ext.Direct calls using Test::ExtDirect
Major application components can be tested with browser automation tools like Selenium.

ARCHITECTURE OVERVIEW

Ext.Direct is not only a transport protocol; in fact it implies a set of requirements for the server side to meet. There are many web server environments available in the Perl world, with several major interface conventions established over the years. These environments are common in the way that all of them implement HTTP request/response model; however the particular details can differ significantly.

To deal with the web server differences, RPC::ExtDirect adopted the core-periphery architecture. Transport layer core, provided by the RPC::ExtDirect CPAN distribution, is complemented by peripheral distributions called gateways that work with particular web server environments. The gateway is responsible for implementing the features missing in a web server interface, if any.

Since the gateway modules implement the "lowest common denominator" abstraction layer, it is fairly easy to use an Ext.Direct application with several different gateways at the same time. The most common use case for this is unit testing; the added benefit is that the application becomes independent of the web server environment and can be ported easily if such a need arises.

See "GATEWAYS" in RPC::ExtDirect for the list of available gateways.

TERMINOLOGY

Ext.Direct uses the following terms, followed by their descriptions:

API

Description of server side calls exposed to client side. API consists of remoting and polling parts.

API declaration

JavaScript chunk that encodes "API". Usually generated by application server and retrieved by client once upon startup; another popular option is to embed API declaration in client side application code at the build time (using Sencha Cmd).

API declaration is generated by RPC::ExtDirect::API module.

Remoting API

The main part of the "API declaration", it declares the Actions and Methods available to the client, as well as their calling conventions, and other parameters.

Polling API

Used to declare the existence of Event Providers and their credentials, basically the URI to use when polling for Events.

Router

Server side component that receives remoting calls, dispatches requests, collects and returns call Results or Exceptions.

Action

Namespace unit; collection of Methods. The nearest Perl analog is a package, other languages may call it a Class.

Since Ext.Direct originated in JavaScript, '::' will be replaced with dots for all Actions in the "API declaration", and should be called as Action.Method() instead of Perl style Action::Method.

Method

Subroutine exposed through Ext.Direct API to be called by client side. Method is fully qualified by "Action" and Method names using a dot as the delimiter: Action.Method.

Method stub

JavaScript function created by the Ext.Direct transport layer on the client side in lieu of the actual "Method" that only exists on the server side. A separate stub will be created for each Method, with the parameter signature conforming to Method's declaration in the "API".

Ordered Method

A "Method" that accepts zero or more parameters in ordered fashion, i.e. by position (in a list). Ordered Methods support call "Metadata".

See more in "METHODS AND CALLING CONVENTIONS" in RPC::ExtDirect.

Named Method

A "Method" that accepts parameters by name, in a hash. Named Methods support call "Metadata".

See more in "METHODS AND CALLING CONVENTIONS" in RPC::ExtDirect.

Form Handler Method

A "Method" that accepts form submits. All form field values are passed to the Form handler in a hash, field => value. One practical reason to use Form handlers is to process file uploads; see FILE UPLOADS for more information. Note also that Form Handler Methods support call "Metadata".

See also "METHODS AND CALLING CONVENTIONS" in RPC::ExtDirect for more information on Form handlers calling convention.

Poll Handler Method

A "Method" that is called by an "Event Provider" to return the list of Events to be passed on to the client side. Poll Handler Methods do not support call "Metadata".

See more in "METHODS AND CALLING CONVENTIONS" in RPC::ExtDirect.

Metadata

A set of extra arguments associated with the Method. Metadata is sent separately from the main arguments and can vary between invocations. Metadata can also follow a calling convention that is different from the main arguments, e.g. it is possible to have "Ordered Method" with Named metadata, and vice versa.

See more in "METHODS AND CALLING CONVENTIONS" in RPC::ExtDirect.

Result

Any data returned by a "Method" upon successful or unsuccessful call completion. This includes application logic errors. 'Not authenticated' and alike events should be returned as Results, not Exceptions.

Exception

Fatal error, or any other unrecoverable event in the application code. Calls that produce Exception instead of "Result" are considered unsuccessful; Ext.Direct provides built in mechanism for managing Exceptions.

Exceptions are not used to indicate errors in application logic flow, only for catastrophic conditions. Nearest analog is status code 500 for HTTP responses.

Examples of Exceptions are: request JSON is broken and can't be decoded; called Method dies because of internal error; Result cannot be encoded in JSON, etc.

Event

An asynchronous notification that can be generated by server side and passed to client side, resulting in some reaction. Events are useful for status updates, progress indicators and other predictably occuring conditions.

Event Provider

Server side script that gets polled by client side every n seconds; default n is 3 but it can be changed in client side configuration.

GETTING STARTED

The first step is to install and configure a gateway that works with your chosen Web server environment. Please refer to gateways' documentation for that. It is recommended to install Test::ExtDirect module as well, so that you could write tests for your Ext.Direct code right from the start.

When you have the gateway configured, it's time to publish some code in the Ext.Direct "API". The easiest way to do this is to use the ExtDirect attribute with the subroutines that need to be published:

package MyApp::Math;

use RPC::ExtDirect Action => 'MyMath';

sub add : ExtDirect(len => 2) {
    my ($class, $a, $b) = @_;
    
    return $a + $b;
}

In this snippet, we have published the MyApp::Math package as an Ext.Direct "Action" called MyMath, and added one "Method" to be exposed to the client side as MyMath.add. In your Ext JS or Sencha Touch app, use the Method with an asynchronous callback function that will be fired when result is transmitted back to the browser:

MyMath.add(a, b, function(result) {
    alert('Multiplication result: ' + result);
});

TESTING YOUR CODE

It is always a good idea to cover your code with unit tests; even more so for something as complex as Remote Procedure Call APIs. A lot of things can go wrong and break the server interface you provide to JavaScript applications. What is worse, these things rarely go wrong upfront, usually the breakage creeps in gradually over time. The only way to ensure that your server side API keeps working exactly as the client side expects it to is via continuous testing.

However testing a server side API with the actual JavaScript code that will consume the API is a daunting task. You would need an instance of the Web server you are going to use in production, a headless Web browser, a ton of infrastructure to keep all this up to date; not even mentioning the time spent maintaining the whole setup. If this picture makes you cringe with frustration, that's totally understandable. But fear not, RPC::ExtDirect has a truly Perlish answer to this problem (making hard things possible).

Test::ExtDirect is the recommended way to unit test your Ext.Direct code. To be precise, it is hardly right to call it unit testing when it involves a test HTTP server, a Perl client that makes actual RPC invocations over HTTP and simulates the JavaScript client; it may be more correct to refer to this process as integration testing instead. However RPC::ExtDirect tries to be completely transparent and never get in the way, so you can think of it as "testing functional units of my code, disregarding the transport".

Supposed that you are sold on the idea, let's see how a unit test for the add subroutine created above looks in practice:

# 01_math.t

use Test::More tests => 1;
use Test::ExtDirect;

use MyApp::Math;

my $result = call_extdirect(
    action => 'MyMath',
    method => 'add',
    arg    => [ 2, 2 ],
);

is $result, 4, "addition result matches";

Now how hard is that?!

GOING FURTHER

Now that we have covered the basics, let's see what else RPC::ExtDirect has in store:

Named parameters

Using named parameters is as easy as ordered ones:

sub named : ExtDirect(params => ['foo', 'bar']) {
    my ($class, %arg) = @_;
    
    my $foo = $arg{foo};
    my $bar = $arg{bar};
    
    # do something, return a scalar
    my $result = ...;
    
    return $result;
}

By default, the named method above will receive only the parameters declared in the ExtDirect attribute. To accept all named parameters, turn off strict checking:

sub named_no_strict
    : ExtDirect(params => ['foo', 'bar'], strict => !1)
{
    my ($class, %arg) = @_;
    
    my $foo = $arg{foo};
    my $bar = $arg{bar};
    my $baz = $arg{baz}; # this parameter is undeclared but gets there
    
    ...
}

It is always a good idea to declare at least some mandatory parameters; however if you want all named arguments to be passed to your Method with no checking, use empty params array:

sub named_no_checks : ExtDirect(params => []) {
    my ($class, %arg) = @_;
    
    my $foo = $arg{foo};
    my $bar = $arg{bar};
    ... # etc
}

Form submits

An Ext.Direct Method can be used to accept form submits in both application/x-www-form-urlencoded and multipart/form-data encodings. This feature can be used to accept file uploads from non-HTML5 browsers (think IE9 and below):

sub handle_upload : ExtDirect(formHandler, upload_arg => 'files') {
    my ($class, %arg) = @_;
    
    my $files = $arg{files}; # arrayref of files
    
    for my $file ( @$files ) {
        ...
    }
}

See more in "FILE UPLOADS" in RPC::ExtDirect.

Handling errors

When your server side code encounters an irrecoverable error, it is a Good Thing to let the client side application know about it. The usual way is to throw an "Exception":

sub dying : ExtDirect(len => 0) { # no parameters
    die "Houston, we've got a problem!\n";
}

Ext.Direct specification requires the server side stack to only send exceptions in debugging mode but never in production mode. The implied security concerns are valid, but having two sets of logic would be unwieldy; RPC::ExtDirect compromises by sending generic exceptions "An error has occured" in production mode (default).

To turn on global debugging, set the debug option in the global API instance's Config:

# Place this in the main app server code
use RPC::ExtDirect;

RPC::ExtDirect->get_api->config->debug(1);

If you are comfortable with exposing internal details of your app server to the outside world even in production mode (and you shouldn't be!), turn on verbose_exceptions:

# This goes to the main app server, too
use RPC::ExtDirect;

RPC::ExtDirect->get_api->config->verbose_exceptions(1);

Be aware that it is an extremely bad idea because it can help malicious parties to find an attack vector to your application!

Using environment objects

Suppose that you want to restrict some parts of the API to be accessible only by authenticated users. The usual way to do this is by using HTTP cookies. However, cookies are not exposed to every Ext.Direct Method by default; you need to tell RPC::ExtDirect to pass an environment object to your Method in order to get low level things like cookies or HTTP request headers:

sub restricted : ExtDirect(params => ['foo'], env_arg => 'env') {
    my ($class, %arg) = @_;
    
    my $env = $arg{env}; # Get the env object
    
    my $user = $env->cookie('user');
    
    # \0 is a shortcut for JSON::false
    return { success => \0, error => 'Not authenticated' }
        unless $user eq 'foo';
    
    ...
}

Using hooks

In the example above, we checked the user's loginedness in the Method that is supposed to do some actual work. Duplicating that code for every restricted Method would be very tedious and error prone; what if we could handle such things in a centralized subroutine that would be called before each restricted Method's invocation and cancel it if the user is not authenticated?

That is what Hooks are for:

sub check_user {
    my ($class, %arg) = @_;
    
    # Hooks always receive an env object
    my $env = $arg{env};
    
    my $user = $env->cookie('user');
    
    # This hashref will be returned to the client side
    # as if the actual Method returned it
    return { success => \0, error => 'Not authenticated' }
        unless $user eq 'foo';
    
    # 1 means we're good to go
    return 1;
}

# This Method won't even be called unless the user is logged in
sub foo : ExtDirect(params => ['foo'], before => \&check_user) {
    my ($class, %arg) = @_;
    
    ...
}

# Same thing, and no code duplication
sub bar : ExtDirect(len => 2, before => \&check_user) {
    my ($class, $a, $b) = @_;
    
    ...
}

In the example above, individual Method level hooks were used; you can also assign hooks to an "Action" (package), or globally. Global hooks are very useful for doing application-wide things like security audit logging:

package MyApp::SecurityLog;

sub log_everything {
    my ($class, %arg) = @_;
    
    my ($method, $result, $ex) = @arg{ qw/ method result exception / };
    
    # Store this into database, or do something else
    ...
}


# This goes to the main program
RPC::ExtDirect->get_api->after('MyApp::SecurityLog::log_everything');

Set this way, log_everything will be called after every Ext.Direct Method invocation, even if a before hook canceled it. See RPC::ExtDirect::API::Hook for the gory details.

Using call metadata

Suppose you want to implement CRUD operations on a set of similar database tables. Instead of declaring a separate set of Methods for each table, you can use call metadata:

package MyApp::Crud;

use RPC::ExtDirect Action => 'MyApp.Database';

# Client will send an array of records as one positional argument
sub create : ExtDirect(1, metadata => {params => ['table'], arg => 9}) {
    my ($class, $records, $meta) = @_;
    
    my $table_name = $meta->{table};
    
    # Insert the records
    ...
}

# Note that for Named methods the default argument is 'metadata'
sub read : ExtDirect(params => [], metadata => {params => ['table']}) {
    my ($class, %arg) = @_;
    
    my $table_name = $arg{metadata}->{table};
    
    # Read the records
    ...
}

# Like with read, a single main argument contains records
sub update : ExtDirect(1, metadata => {params => ['table'], arg => 9}) {
    my ($class, $records, $meta) = @_;
    
    my $table_name = $meta->{table};
    
    # Update the records
    ...
}

# Again a single main argument with array of records
sub delete : ExtDirect(1, metadata => {params => ['table'], arg => 9}) {
    my ($class, $records, $meta) = @_;
    
    my $table_name = $meta->{table};
    
    # Delete the records
    ...
}

This can now be used easily on the client side (with Ext JS 5.1+):

var grid = new Ext.grid.Panel({
    title: 'Ext.Direct grid',
    
    store: {
        fields: ['frobbe', 'throbbe'],
        proxy: {
            type: 'direct',
            api: {
                create:  'MyApp.Database.create',
                read:    'MyApp.Database.read',
                update:  'MyApp.Database.update',
                destroy: 'MyApp.Database.delete'
            },
            metadata: {
                table: 'foo'
            }
        }
    },
    
    columns: [{
        text: 'Frobbe',
        dataIndex: 'frobbe'
    }, {
        text: 'Throbbe',
        dataIndex: 'throbbe'
    }],
    
    ...
});

It may be also worth noting that RPC::ExtDirect::Client supports call metadata fully as of version 1.10.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

DESCRIPTION

What is Ext.Direct?

What Ext.Direct is for?

ARCHITECTURE OVERVIEW

TERMINOLOGY

GETTING STARTED

TESTING YOUR CODE

GOING FURTHER

Named parameters

Form submits

Handling errors

Using environment objects

Using hooks

Using call metadata

SEE ALSO

Module Install Instructions