bilma-cls-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "770a3433-9fc5-4e65-b966-0da5a1d7b578",
   "metadata": {},
   "source": [
    "# Bilma Classification demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1050cd58-3c89-41d1-9612-ac77af0ac27a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bilma import bilma_model\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d92a384-7f73-4f4c-a068-37515362faa7",
   "metadata": {},
   "source": [
    "## Load the model\n",
    "\n",
    "You can load the model using `bilma_model.load`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "de4ee967-8859-4fdf-a2c2-f0f319236503",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = bilma_model.load(\"models-final/bilma_small_MX_epoch-1_classification_epochs-13.h5\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21584d42-5afe-40eb-9e9b-f3677f0d2811",
   "metadata": {},
   "source": [
    "The layers and parameters of the model are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dc43b5f8-7ce7-4740-aafa-2c6551cbb446",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "capt_input (InputLayer)         [(None, 280)]        0                                            \n",
      "__________________________________________________________________________________________________\n",
      "embedding (Embedding)           (None, 280, 512)     14860800    capt_input[0][0]                 \n",
      "__________________________________________________________________________________________________\n",
      "encoder_5 (Encoder)             (None, 280, 512)     9456640     embedding[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "tf.__operators__.getitem (Slici (None, 1, 512)       0           encoder_5[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "tf.compat.v1.squeeze (TFOpLambd (None, 512)          0           tf.__operators__.getitem[0][0]   \n",
      "__________________________________________________________________________________________________\n",
      "ex_1 (Dense)                    (None, 512)          262656      tf.compat.v1.squeeze[0][0]       \n",
      "__________________________________________________________________________________________________\n",
      "ex_2 (Dense)                    (None, 512)          262656      ex_1[0][0]                       \n",
      "__________________________________________________________________________________________________\n",
      "dense_37 (Dense)                (None, 280, 29025)   14889825    encoder_5[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "cp (Dense)                      (None, 15)           7695        ex_2[0][0]                       \n",
      "==================================================================================================\n",
      "Total params: 39,740,272\n",
      "Trainable params: 39,740,272\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1f704cd-5db3-45ce-a592-813c4d5c9599",
   "metadata": {},
   "source": [
    "The input is an array of shape `(bs, 280)` where bs is the batch size and 280 is the max lenght of the sequences. \n",
    "\n",
    "The outputs have shape `(bs, 280, 29025)` and  `(bs, 15)`. The first output is the same as the MLM model. The second output predicts the probability of each of the 15 emoticons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "717dd9c5-3ddd-4981-a98a-d59a6179c6f8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([<KerasTensor: shape=(None, 280) dtype=float32 (created by layer 'capt_input')>],\n",
       " [<KerasTensor: shape=(None, 280, 29025) dtype=float32 (created by layer 'dense_37')>,\n",
       "  <KerasTensor: shape=(None, 15) dtype=float32 (created by layer 'cp')>])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.inputs, model.outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a157c06-160e-43dd-b4f6-d5c04988f369",
   "metadata": {},
   "source": [
    "To input data into the model we need a tokenizer just like in the MLM model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "29e2c47a-fb7f-4853-8728-bf41c63134c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = bilma_model.tokenizer(vocab_file=\"d:/data/twitts/vocab_file_ALL.txt\", max_length=280)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bea7c297-45e5-4ac1-a624-1b55f91ce365",
   "metadata": {},
   "outputs": [],
   "source": [
    "texts = [\"Tenemos tres días sin internet ni señal de celular en el pueblo.\",\n",
    "         \"Incomunicados en el siglo XXI tampoco hay servicio de telefonía fija\",\n",
    "         \"Vamos a comer unos tacos\",\n",
    "         \"Los del banco no dejan de llamarme\"]\n",
    "tweet = tokenizer.tokenize(texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9698de12-1aaa-489b-9db4-b76267857400",
   "metadata": {},
   "outputs": [],
   "source": [
    "We can now predict the emoticons "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "29fa9a1c-46e3-4c9e-a2fa-b6c74caaa368",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 15)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = model.predict(tweet)\n",
    "p[1].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0f8c70d2-e177-4e01-a693-5009ed208722",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['🥺', '🤔', '😍', '😡']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.decode_emo(p[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ad34247-8685-478b-9a18-dde7338b7ecc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}