NAME

RDFStore::Parser::SiRPAC - This module implements a streaming RDF Parser as a direct implementation of XML::Parser::Expat(3)

SYNOPSIS

	use RDFStore::Parser::SiRPAC;
        use RDFStore::NodeFactory;
        my $p=new RDFStore::Parser::SiRPAC(
		ErrorContext => 2,
                Handlers        => {
                        Init    => sub { print "INIT\n"; },
                        Final   => sub { print "FINAL\n"; },
                        Assert  => sub { print "STATEMENT - @_\n"; }
                },
                NodeFactory     => new RDFStore::NodeFactory() );

	$p->parsefile('http://www.gils.net/bsr-gils.rdfs');
        $p->parsefile('http://www.gils.net/rdf/bsr-gils.rdfs');
        $p->parsefile('/some/where/my.rdf');
        $p->parsefile('file:/some/where/my.rdf');
	$p->parse(*STDIN);

	use RDFStore;
	my $pstore=new RDFStore::Parser::SiRPAC(
                ErrorContext 	=> 2,
                Style 		=> 'RDFStore::Parser::Styles::MagicTie',
                NodeFactory     => new RDFStore::NodeFactory(),
                Source  	=> 'http://www.gils.net/bsr-gils.rdfs',
                store   =>      {
                                	persistent      =>      1,
                                	directory       =>      '/tmp/',
                                	seevalues       =>      1,
                                	options         =>      { style => 'BerkeleyDB', Q => 20 }
                                }
        );
	$pstore->parsefile('http://www.gils.net/bsr-gils.rdfs');

	#using the Expat no-blocking feature
	my $nbpstore = $pstore->parse_start();
	while (<STDIN>) {
		$nbpstore->parse_more($_);
	};	
	$nbpstore->parse_done();
	

DESCRIPTION

This module implements a Resource Description Framework (RDF) streaming parser completely in Perl using the XML::Parser::Expat(3) module. The actual RDF parsing happens using an instance of XML::Parser::Expat with Namespaces option enabled and start/stop and char handlers set. The RDF specific code is based on the modified version of SiRPAC of Sergey Melnik in Java; a lot of changes and adaptations have been done to actually run it under Perl. Expat options may be provided when the RDFStore::Parser::SiRPAC object is created. These options are then passed on to the Expat object on each parse call.

Exactly like XML::Parser(3) the behavior of the parser is controlled either by the Style entry elsewhere in this document and/or the Handlers entry elsewhere in this document options, or by the setHandlers entry elsewhere in this document method. These all provide mechanisms for RDFStore::Parser::SiRPAC to set the handlers needed by Expat. If neither Style nor Handlers are specified, then parsing just checks the RDF document syntax against the W3C RDF Raccomandation . When underlying handlers get called, they receive as their first parameter the Expat object, not the Parser object.

To see some examples about how to use it look at the sections below and in the samples and utils directory coming with this software distribution.

E.g. With RDFStore::Parser::SiRPAC you can easily write an rdfingest.pl script to do something like this:

fetch -o - -q http://dmoz.org/rdf/content.rdf.u8.gz | \
	gunzip - | \
	sed -f dmoz.content.sed | rdfingest.pl - 

METHODS

new

This is a class method, the constructor for RDFStore::Parser::SiRPAC. Options are passed as keyword value pairs. Recognized options are:

• NodeFactory

  This option is mandatory to run the RDFStore::Parser::SiRPAC parser correctly and must contain a reference to an object of type RDFStore::Stanford::NodeFactory(3). Such a reference is used during the RDF parsing to create resources, literal and statements to be passed to the registered handlers. A sample implementation is RDFStore::NodeFactory that is provided with the RDFStore package.

• Source

  This option can be specified by the user to set a base URI to use for the generation of resource URIs during parsing. If this option is omitted the parser will try to generate a prefix for generated resources using the input filename or URL actually containing the input RDF. In a near future such an option could be obsoleted by use of XMLBase W3C raccomandation.

• GenidNumber

  Seed the genid numbers with the given value

• bCreateBags

  Flag to generate a Bag for each Description element

• Style

  This option provides an easy way to set a given style of parser. There is one sample Sylte module provided with the RDFStore::Parser::SiRPAC distribution called RDFStore::Parser::SiRPAC::RDFStore. Such a module uses the RDFStore(3) modules together with the Data::MagicTie(3) to implement a simple RDF storage. Custom styles can be provided by giving a full package name containing at least one '::'. This package should then have subs defined for each handler it wishes to have installed. See "WRITE YOUR OWN PARSER" below for a discussion on how to build one.

• Handlers

  When provided, this option should be an anonymous hash containing as keys the type of handler and as values a sub reference to handle that type of event. All the handlers get passed as their 1st parameter the instance of Expat that is parsing the document. Further details on handlers can be found in "HANDLERS". Any handler set here overrides the corresponding handler set with the Style option.

• ErrorContext

  This is an XML::Parser option. When this option is defined, errors are reported in context. The value should be the number of lines to show on either side of the line in which the error occurred.

All the other XML::Parser and XML::Parser::Expat options should work freely with RDFStore::Parser::SiRPAC see XML::Parser(3) and XML::Parser::Expat(3).

setHandlers(TYPE, HANDLER [, TYPE, HANDLER [...]])

This method registers handlers for various parser events. It overrides any previous handlers registered through the Style or Handler options or through earlier calls to setHandlers. By providing a false or undefined value as the handler, the existing handler can be unset.

This method returns a list of type, handler pairs corresponding to the input. The handlers returned are the ones that were in effect prior to the call.

See a description of the handler types in "HANDLERS".

parse(SOURCE, URIBASE [, OPT => OPT_VALUE [...]])

The SOURCE parameter should either be a string containing the whole RDF document, or it should be an open IO::Handle. The URIBASE can be specified by the user to set a base URI to use for the generation of resource URIs during parsing. If this option is omitted the parser will try to generate a prefix for generated resources using either the Source option of the constructor, the input filename or URL actually containing the input RDF. In a near future such an option could be obsoleted by use of XMLBase W3C raccomandation. Constructor options to XML::Parser::Expat given as keyword-value pairs may follow the URIBASE parameter. These override, for this call, any options or attributes passed through from the RDFStore::Parser::SiRPAC instance.

A die call is thrown if a parse error occurs. Otherwise it will return 1 or whatever is returned from the Final handler, if one is installed. In other words, what parse may return depends on the style.

e.g. the RDFStore::Parser::SiRPAC::RDFStore Style module returns an instance of RDFStore::Stanford::Model

parsestring(STRING, URIBASE [, OPT => OPT_VALUE [...]])

This is just an alias for parse for backwards compatibility.

parsefile(URL_OR_FILE [, OPT => OPT_VALUE [...]])

Open URL_OR_FILE for reading, then call parse with the open handle. If URL_OR_FILE is a full qualified URL this module uses Socket(3) to actually fetch the content. The URIBASE parse() parameter is set to URL_OR_FILE.

getReificationCounter()

Return the latest genid number generated by the parser

HANDLERS

As Expat, SiRPAC is an event based parser. As the parser recognizes parts of the RDF document then any handlers registered for that type of an event are called with suitable parameters. All handlers receive an instance of XML::Parser::Expat as their first argument. See "METHODS" in XML::Parser::Expat for a discussion of the methods that can be called on this object.

Init (Expat)

This is called just before the parsing of the document starts.

Final (Expat)

This is called just after parsing has finished, but only if no errors occurred during the parse. Parse returns what this returns.

Assert (Expat, Statement)

This event is generated when a new RDF statement has been generated by the parseer.start tag is recognized. Statement is of type RDFStore::Stanford::Statement(3) as generated by the RDFStore::Stanford::NodeFactory(3) passed as argument to the RDFStore::Parser::SiRPAC constructor.

Start_XML_Literal (Expat, Element [, Attr, Val [,...]])

This event is generated when an XML start tag is recognized within an RDF property with parseType="Literal". Element is the name of the XML element type that is opened with the start tag. The Attr & Val pairs are generated for each attribute in the start tag.

This handler should return a string containing either the original XML chunck or one f its transformations, perhaps using XSLT.

Stop_XML_Literal (Expat, Element)

This event is generated when an XML end tag is recognized within an RDF property with parseType="Literal". Note that an XML empty tag (<foo/>) generates both a Start_XML_Literal and an Stop_XML_Literal event.

Char_XML_Literal (Expat, String)

This event is generated when non-markup is recognized within an RDF property with parseType="Literal". The non-markup sequence of characters is in String. A single non-markup sequence of encoding of the string in the original document, this is given to the handler in UTF-8.

This handler should return the processed text as a string.

WRITE YOUR OWN PARSER

Write an extension module for you needs it is as easy as write one for XML::Parser :) Have a look at http://www.xml.com/xml/pub/98/09/xml-perl.html and http://wwwx.netheaven.com/~coopercc/xmlparser/intro.html.

You can either make you Perl script a parser self by embedding the needed function hooks or write a custom Style module for RDFStore::Parser::SiRPAC.

*.pl scripts

use RDFStore::Parser::SiRPAC;
use RDFStore::NodeFactory;
my $p=new RDFStore::Parser::SiRPAC(
	Handlers        => {
		Init    => sub { print "INIT\n"; },
		Final   => sub { print "FINAL\n"; },
		Assert  => sub { print "STATEMENT - @_\n"; }
	},
	NodeFactory     => new RDFStore::NodeFactory() );

or something like:

	use RDFStore::Parser::SiRPAC;
        use RDFStore::NodeFactory;
	my $p=new RDFStore::Parser::SiRPAC( NodeFactory     => new RDFStore::NodeFactory() );
	$p->setHandlers(        Init    => sub { print "INIT\n"; },
                        	Final   => sub { print "FINAL\n"; },
                        	Assert  => sub { print join(",",@_),"\n"; }     );

Style modules

A more sophisticated solution is to write a complete Perl5 Sytle module for RDFStore::Parser::SiRPAC that can be easily reused in your code. E.g. a perl script could use this piece of code:

	use RDFStore::Parser::SiRPAC;
	use RDFStore::Parser::SiRPAC::MyStyle;
	use RDFStore::NodeFactory;

	my $p=new RDFStore::Parser::SiRPAC(	Style => 'RDFStore::Parser::SiRPAC::MyStyle',
                			NodeFactory     => new RDFStore::NodeFactory() );
	$p->parsefile('http://www.gils.net/bsr-gils.rdfs');

The Style module self could stored into a file like MyStyle.pm like this:

	package RDFStore::Parser::SiRPAC::MyStyle;

	sub Init { print "INIT\n"; };
	sub Final { print "FINAL\n"; };
	sub Assert {
                print "ASSERT: ",
                                $_[1]->subject()->toString(),
                                $_[1]->predicate()->toString(),
                                $_[1]->object()->toString(), "\n";
	};
	sub Start_XML_Literal { print "STARTAG: ",$_[1],"\n"; };
	sub Stop_XML_Literal { print "ENDTAG: ",$_[1],"\n"; };
	sub Char_XML_Literal { print "UTF8 chrs: ",$_[1],"\n"; };

	1;

For a more complete and useful example see RDFStore::Parser::SiRPAC::RDFStore(3).

BUGS

This module implements most of the W3C RDF Raccomandation as its Java counterpart SiRPAC from the Stanford University Database Group by Sergey Melnik (see http://www-db.stanford.edu/~melnik/rdf/api.html) This version is conformant to the latest RDF API Draft on 2000-11-13. It does not support yet:

* aboutEach and aboutEachPrefix

SEE ALSO

RDFStore::Parser::SiRPAC(3), DBMS(3) and XML::Parser(3) XML::Parser::Expat(3) RDFStore::Stanford::Model(3) RDFStore::NodeFactory(3)

RDF Model and Syntax Specification - http://www.w3.org/TR/REC-rdf-syntax
RDF Schema Specification 1.0 - http://www.w3.org/TR/2000/CR-rdf-schema-20000327
Benchmarking XML Parsers by Clark Cooper - http://www.xml.com/pub/Benchmark/article.html
See also http://www.w3.org/RDF/Implementations/SiRPAC/SiRPAC-defects.html
RDF::Parser(3) from http://www.pro-solutions.com

AUTHOR

Alberto Reggiori <areggiori@webweaving.org>

Sergey Melnik <melnik@db.stanford.edu> is the original author of the streaming version of SiRPAC in Java
Clark Cooper is the author of the XML::Parser(3) module together with Larry Wall

cpanm RDFStore

perl -MCPAN -e shell
install RDFStore