NAME

AI::MXNet::RNN::IO - Functions for constructing recurrent neural networks.

SYNOPSIS

DESCRIPTION

Functions for constructing recurrent neural networks.

Encode sentences and (optionally) build a mapping
from string tokens to integer indices. Unknown keys
will be added to vocabulary.

Parameters
----------
sentences : array ref of array refs of str
    A array ref of sentences to encode. Each sentence
    should be a array ref of string tokens.
vocab : undef or hash ref of str -> int
    Optional input Vocabulary
invalid_label : int, default -1
    Index for invalid token, like <end-of-sentence>
invalid_key : str, default '\n'
    Key for invalid token. Use '\n' for end
    of sentence by default.
start_label : int
    lowest index.

Returns
-------
result : array ref of array refs of int
    encoded sentences
vocab : hash ref of str -> int
    result vocabulary

NAME

AI::MXNet::BucketSentenceIter

SYNOPSIS

DESCRIPTION

Simple bucketing iterator for language model.
Label for each step is constructed from data of
next step.

new

Parameters
----------
sentences : array ref of array refs of int
    encoded sentences
batch_size : int
    batch_size of data
invalid_label : int, default -1
    key for invalid label, e.g. <end-of-sentence>
dtype : str, default 'float32'
    data type
buckets : array ref of int
    size of data buckets. Automatically generated if undef.
data_name : str, default 'data'
    name of data
label_name : str, default 'softmax_label'
    name of label
layout : str
    format of data and label. 'NT' means (batch_size, length)
    and 'TN' means (length, batch_size).