Convolutional Network

A Convolutional Network can be used for training a classifier on sentences. For example such network can be used for sentiment classification.

The network consists of the following layers:

1. Lookup layer: extracts features from sentences using a list of extractors
2. Linear layer
3. Max out layer
4. Hardtanh layer
5. Linear layer

The network can be trained on a corpus of annotated sentences in the format of SemEval 2013 Task 2 on Sentiment Analysis in Twitter, i.e.

• one sentence per line
• tab separated fields. The polarity field is the third one and the text is in the fourth.

For example: SID UID polarity tokenized text 264183816548130816 15140428 positive Gas by my house hit $3.39!!!! I'm going to Chapel Hill on Sat. :)

The polarity field can be set with an option below.

The script to train a convolutional network or use it for classification is dl-conv.py.

Usage

The script dl-conv.py can be invoked like this:

dl-conv.py [-h] [-c FILE] [-t TRAIN] [-w WINDOW] [-s EMBEDDINGS_SIZE]
              [-e ITERATIONS] [-l LEARNING_RATE] [-n HIDDEN] [-n2 HIDDEN2]
              [--caps [CAPS]] [--suffix [SUFFIX]] [--suffixes SUFFIXES]
              [--prefix [PREFIX]] [--prefixes PREFIXES] [--vocab VOCAB]
              [--vectors VECTORS] [--min-occurr MINOCCURR] [--load LOAD]
              [--variant VARIANT] [--threads THREADS] [-v]
              model

Convolutional network classifier.

positional arguments:
  model                 Model file to train/use.

optional arguments:
  -h, --help            show this help message and exit
  -c FILE, --config FILE
                    Specify config file
  -t TRAIN, --train TRAIN
                    File with annotated data for training.
  --threads THREADS     Number of threads (default 1)
  -v, --verbose         Verbose mode

Train:
  -w WINDOW, --window WINDOW
                    Size of the word window (default 5)
  -s EMBEDDINGS_SIZE, --embeddings-size EMBEDDINGS_SIZE
                    Number of features per word (default 50)
  -e ITERATIONS, --epochs ITERATIONS
                    Number of training epochs (default 100)
  -l LEARNING_RATE, --learning_rate LEARNING_RATE
                    Learning rate for network weights (default 0.001)
  -n HIDDEN, --hidden HIDDEN
                    Number of hidden neurons (default 200)
  -n2 HIDDEN2, --hidden2 HIDDEN2
                    Number of hidden neurons (default 200)

Extractors:
  --caps [CAPS]         Include capitalization features. Optionally, supply
                    the number of features (default 5)
  --suffix [SUFFIX]     Include suffix features. Optionally, supply the number
                    of features (default 5)
  --suffixes SUFFIXES   Load suffixes from this file
  --prefix [PREFIX]     Include prefix features. Optionally, supply the number
                    of features (default 0)
  --prefixes PREFIXES   Load prefixes from this file

Embeddings:
  --vocab VOCAB         Vocabulary file, either read or created
  --vectors VECTORS     Embeddings file, either read or created
  --min-occurr MINOCCURR
                    Minimum occurrences for inclusion in vocabulary
  --load LOAD           Load previously saved model
  --variant VARIANT     Either "senna" (default), "polyglot" or "word2vec".