NAME
AI::MXNet::RNN::IO - Functions for constructing recurrent neural networks.
SYNOPSIS
DESCRIPTION
Functions for constructing recurrent neural networks.
Encode sentences and (optionally) build a mapping from string tokens to integer indices. Unknown keys will be added to vocabulary.
Parameters ---------- sentences : array ref of array refs of str A array ref of sentences to encode. Each sentence should be a array ref of string tokens. vocab : undef or hash ref of str -> int Optional input Vocabulary invalid_label : int, default -1 Index for invalid token, like <end-of-sentence> invalid_key : str, default '\n' Key for invalid token. Use '\n' for end of sentence by default. start_label : int lowest index.
Returns ------- result : array ref of array refs of int encoded sentences vocab : hash ref of str -> int result vocabulary
NAME
AI::MXNet::BucketSentenceIter
SYNOPSIS
DESCRIPTION
Simple bucketing iterator for language model. Label for each step is constructed from data of next step.
new
Parameters ---------- sentences : array ref of array refs of int encoded sentences batch_size : int batch_size of data invalid_label : int, default -1 key for invalid label, e.g. <end-of-sentence> dtype : str, default 'float32' data type buckets : array ref of int size of data buckets. Automatically generated if undef. data_name : str, default 'data' name of data label_name : str, default 'softmax_label' name of label layout : str format of data and label. 'NT' means (batch_size, length) and 'TN' means (length, batch_size).