Pandas_task_solution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Дополнительный материал: библиотека Pandas\n",
    "\n",
    "Библиотека Pandas - важная библиотека, актиавно используемая специалистами по анализу данных. Pandas  --- это модуль для первичной работы с данными, с помощью которого можно проводить простой анализ данных и предобработку данных.\n",
    "Основные возможности:\n",
    "* удобное чтение и запись данных из csv, txt, xls, SQL databases, HDF5\n",
    "* широкий функционал для выполнения преобразований данных\n",
    "* удобная работа с пропусками в данных\n",
    "* поиск, сортировка, выборка объектов, удовлетворяющих заданным критериям\n",
    "* возможности по соединению датасетов\n",
    "* красивая визуализация"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Загрузка данных и создание датафреймов"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Наиболее популярные форматы данных (при скачивании датасета из интернета)__:\n",
    "* _csv_ (comma separated file), _tsv_ (tab separated file) - таблицы, записанные в текстовые файлы с простой структурой. Эти файлы можно открывать в обычном текстовом редакторе. Pandas позволяет считывать эти данные именно в формате таблицы.\n",
    "* _xls_ (eXceL Spreadsheet ---таблицы Microsoft)\n",
    "* _json_ (JavaScript Object Notation, используется для _сериализации_ структур языка, то есть сохранения сложных объектов, например, вложенных списков или словарей python). Json-текст представляет собой либо набор пар ключ: значение, либо упорядоченный набор значений\n",
    "\n",
    "В pandas есть функции для считывания во всех этих форматах.\n",
    "\n",
    "__Чтение из csv с помощью pandas__:\n",
    "[pandas.read_csv](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html).\n",
    "У функции  параметров, основные необходимые:\n",
    "* filepath_or_buffer (перый и единственный обязательный аргумент) --- имя файла\n",
    "* sep -- разделитель (; , \\t ...)\n",
    "* quotechar --- символ кавычек, все что внутри считается за строку (разделители также могут входить в эту строку; ' \" ...)\n",
    "* names --- список названий колонок\n",
    "* header --- номер строки файла (с 0), которую нужно считать заголовком\n",
    "* dtype --- словарь, сопоставляющий именам колонок типы данных в них\n",
    "* na_values --- строка/список/словарь (ключи -- названия колонок) строковых значений, которые нужно считать пропуском.\n",
    "\n",
    "По умолчанию names=None и header=0, то есть названия колонок берутся из первой строки файла. Можно передать названия через names. Если вы не хотите давать названия, укажите header=None, тогда названия будут даны автоматически индексами с 0. Учтите, что названия нужны при дальнейшей работе с данными. Следите за длиной списка названий, он должен совпадать с реальным числом колонок в файле (а в противном случае вы не получите ошибки)! Чтобы заменить заголовки, записанные в файле, нужно установить header=0 и передать names.\n",
    "\n",
    "В функцию pd.read\\_csv можно передавать как путь к файлу, хранящемуся на компьютере, так и ссылку на файл в Интернете.\n",
    "\n",
    "Для чтения xls: [pandas.read_excel](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_excel.html)\n",
    "\n",
    "Для чтения sql: [pandas.read_sql](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_sql.html)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Считывание данных:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv(\"https://raw.githubusercontent.com/iad34/seminars/master/materials/data_sem1.csv\",\n",
    "                   sep=\";\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Мы будем работать с данными Titanic: он содержит информаицию о пассажирах корабля, включая их демографические характеристики и выжил пассажир или нет."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Посмотреть первые строки:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings; Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen; Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund; Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings; Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen; Miss. Laina  female  26.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Переменная, которую возвращает функция чтения, ссылается на _датафрейм_; это основная структура данных в pandas. Его можно создать и вручную:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AAA</th>\n",
       "      <th>BBB</th>\n",
       "      <th>CCC</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>20</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>30</td>\n",
       "      <td>-30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>40</td>\n",
       "      <td>-50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   AAA  BBB  CCC\n",
       "0    4   10  100\n",
       "1    5   20   50\n",
       "2    6   30  -30\n",
       "3    7   40  -50"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({'AAA' : [4,5,6,7], 'BBB' : [10,20,30,40], 'CCC' : [100,50,-30,-50]})\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Работа со строками и столбцами датафрейма"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Датафрейм - это таблица. Названия строк:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=891, step=1)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Названия столбов:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp',\n",
       "       'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Полезный функционал:\n",
    "* параметр df.dtypes --- типы колонок\n",
    "* метод [df.fillna(value)](http://pandas.pydata.org/pandas-docs/version/0.17.1/generated/pandas.DataFrame.fillna.html) --- заменить пропущенные значения, value --- на что заменить (число или словарь с ключами-названиями колонок)\n",
    "* методы df.head([N]) и df.tail([N]) --- показать N (необязательный аргумент) первых или последних значений\n",
    "* параметры df.index, df.columns и df.values --- соответственно названия строк датафрейма, названия колонок и таблица данных.\n",
    "* метод df.T --- транспонировать данные (поменять строки и столбцы местами)\n",
    "* сортировка данных по индексу (по названиям строк) и по значениям колонок, например df.sort_index(axis=1, ascending=False) и df.sort_values(by='B')\n",
    "* метод df.copy() --- копировать датафрейм\n",
    "\n",
    "\n",
    "В датафрейме могут храниться данные разных типов (главное, чтобы тип был один и тот же внутри колонки)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Выбор нескольких столбцов:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Cabin</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>22.0</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>38.0</td>\n",
       "      <td>female</td>\n",
       "      <td>C85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>26.0</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>35.0</td>\n",
       "      <td>female</td>\n",
       "      <td>C123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35.0</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Age     Sex Cabin\n",
       "0  22.0    male   NaN\n",
       "1  38.0  female   C85\n",
       "2  26.0  female   NaN\n",
       "3  35.0  female  C123\n",
       "4  35.0    male   NaN"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[[\"Age\", \"Sex\", \"Cabin\"]].head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PassengerId\n"
     ]
    }
   ],
   "source": [
    "# можно \"доставать\" названия колонок\n",
    "print(data.columns[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PassengerId      int64\n",
      "Survived         int64\n",
      "Pclass           int64\n",
      "Name            object\n",
      "Sex             object\n",
      "Age            float64\n",
      "SibSp            int64\n",
      "Parch            int64\n",
      "Ticket          object\n",
      "Fare           float64\n",
      "Cabin           object\n",
      "Embarked        object\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "# типы столбцов\n",
    "print(data.dtypes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В машинном обучении библиотеку удобно использовать со следующей интерпретацией: по строкам датафрейма находятся объекты, по столбцам - признаки и целевая переменная. В нашем датафрейме целевая переменная задана в столбце Survived. Если мы бы хотели выделить часть датафрейма без этого столбца, мы бы использовали такой код:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings; Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen; Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle; Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen; Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                               Name  \\\n",
       "0            1       3                            Braund; Mr. Owen Harris   \n",
       "1            2       1  Cumings; Mrs. John Bradley (Florence Briggs Th...   \n",
       "2            3       3                             Heikkinen; Miss. Laina   \n",
       "3            4       1       Futrelle; Mrs. Jacques Heath (Lily May Peel)   \n",
       "4            5       3                           Allen; Mr. William Henry   \n",
       "\n",
       "      Sex   Age  SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0    male  22.0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1  female  38.0      1      0          PC 17599  71.2833   C85        C  \n",
       "2  female  26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3  female  35.0      1      0            113803  53.1000  C123        S  \n",
       "4    male  35.0      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.drop(\"Survived\", axis=1).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обратите внимание, что столбец не удалился из датафрейма навсегда. Наоборот, результат нашей операции записан в выводе, и если бы мы хотели его сохранить, мы должны были бы присвоить результат операции новой переменной."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Индексация по строкам (выбрать все строки с четными индексами):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "data2 = data.iloc[data.index % 2 == 0].head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Использовать столбец Name для задания названий строк (index):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.set_index(data[\"Name\"], inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Braund; Mr. Owen Harris</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cumings; Mrs. John Bradley (Florence Briggs Thayer)</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings; Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Heikkinen; Miss. Laina</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen; Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Futrelle; Mrs. Jacques Heath (Lily May Peel)</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle; Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Allen; Mr. William Henry</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen; Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    PassengerId  Survived  \\\n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                                       1         0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...            2         1   \n",
       "Heikkinen; Miss. Laina                                        3         1   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)                  4         1   \n",
       "Allen; Mr. William Henry                                      5         0   \n",
       "\n",
       "                                                    Pclass  \\\n",
       "Name                                                         \n",
       "Braund; Mr. Owen Harris                                  3   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...       1   \n",
       "Heikkinen; Miss. Laina                                   3   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             1   \n",
       "Allen; Mr. William Henry                                 3   \n",
       "\n",
       "                                                                                                 Name  \\\n",
       "Name                                                                                                    \n",
       "Braund; Mr. Owen Harris                                                       Braund; Mr. Owen Harris   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  Cumings; Mrs. John Bradley (Florence Briggs Th...   \n",
       "Heikkinen; Miss. Laina                                                         Heikkinen; Miss. Laina   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             Futrelle; Mrs. Jacques Heath (Lily May Peel)   \n",
       "Allen; Mr. William Henry                                                     Allen; Mr. William Henry   \n",
       "\n",
       "                                                       Sex   Age  SibSp  \\\n",
       "Name                                                                      \n",
       "Braund; Mr. Owen Harris                               male  22.0      1   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  female  38.0      1   \n",
       "Heikkinen; Miss. Laina                              female  26.0      0   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        female  35.0      1   \n",
       "Allen; Mr. William Henry                              male  35.0      0   \n",
       "\n",
       "                                                    Parch            Ticket  \\\n",
       "Name                                                                          \n",
       "Braund; Mr. Owen Harris                                 0         A/5 21171   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...      0          PC 17599   \n",
       "Heikkinen; Miss. Laina                                  0  STON/O2. 3101282   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)            0            113803   \n",
       "Allen; Mr. William Henry                                0            373450   \n",
       "\n",
       "                                                       Fare Cabin Embarked  \n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                              7.2500   NaN        S  \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  71.2833   C85        C  \n",
       "Heikkinen; Miss. Laina                               7.9250   NaN        S  \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        53.1000  C123        S  \n",
       "Allen; Mr. William Henry                             8.0500   NaN        S  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь строки индексируются с помощью Name (выделено жирным в начале каждой строки)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Считывание данных без названий колонок и придумывание своих названий колонкам:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_m = pd.read_csv(\"https://archive.ics.uci.edu/ml/machine-learning-databases/mushroom/agaricus-lepiota.data\",\n",
    "                     header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_m.columns = [\"col\"+str(i) for i in range(23)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col0</th>\n",
       "      <th>col1</th>\n",
       "      <th>col2</th>\n",
       "      <th>col3</th>\n",
       "      <th>col4</th>\n",
       "      <th>col5</th>\n",
       "      <th>col6</th>\n",
       "      <th>col7</th>\n",
       "      <th>col8</th>\n",
       "      <th>col9</th>\n",
       "      <th>...</th>\n",
       "      <th>col13</th>\n",
       "      <th>col14</th>\n",
       "      <th>col15</th>\n",
       "      <th>col16</th>\n",
       "      <th>col17</th>\n",
       "      <th>col18</th>\n",
       "      <th>col19</th>\n",
       "      <th>col20</th>\n",
       "      <th>col21</th>\n",
       "      <th>col22</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>p</td>\n",
       "      <td>x</td>\n",
       "      <td>s</td>\n",
       "      <td>n</td>\n",
       "      <td>t</td>\n",
       "      <td>p</td>\n",
       "      <td>f</td>\n",
       "      <td>c</td>\n",
       "      <td>n</td>\n",
       "      <td>k</td>\n",
       "      <td>...</td>\n",
       "      <td>s</td>\n",
       "      <td>w</td>\n",
       "      <td>w</td>\n",
       "      <td>p</td>\n",
       "      <td>w</td>\n",
       "      <td>o</td>\n",
       "      <td>p</td>\n",
       "      <td>k</td>\n",
       "      <td>s</td>\n",
       "      <td>u</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>e</td>\n",
       "      <td>x</td>\n",
       "      <td>s</td>\n",
       "      <td>y</td>\n",
       "      <td>t</td>\n",
       "      <td>a</td>\n",
       "      <td>f</td>\n",
       "      <td>c</td>\n",
       "      <td>b</td>\n",
       "      <td>k</td>\n",
       "      <td>...</td>\n",
       "      <td>s</td>\n",
       "      <td>w</td>\n",
       "      <td>w</td>\n",
       "      <td>p</td>\n",
       "      <td>w</td>\n",
       "      <td>o</td>\n",
       "      <td>p</td>\n",
       "      <td>n</td>\n",
       "      <td>n</td>\n",
       "      <td>g</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>e</td>\n",
       "      <td>b</td>\n",
       "      <td>s</td>\n",
       "      <td>w</td>\n",
       "      <td>t</td>\n",
       "      <td>l</td>\n",
       "      <td>f</td>\n",
       "      <td>c</td>\n",
       "      <td>b</td>\n",
       "      <td>n</td>\n",
       "      <td>...</td>\n",
       "      <td>s</td>\n",
       "      <td>w</td>\n",
       "      <td>w</td>\n",
       "      <td>p</td>\n",
       "      <td>w</td>\n",
       "      <td>o</td>\n",
       "      <td>p</td>\n",
       "      <td>n</td>\n",
       "      <td>n</td>\n",
       "      <td>m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>p</td>\n",
       "      <td>x</td>\n",
       "      <td>y</td>\n",
       "      <td>w</td>\n",
       "      <td>t</td>\n",
       "      <td>p</td>\n",
       "      <td>f</td>\n",
       "      <td>c</td>\n",
       "      <td>n</td>\n",
       "      <td>n</td>\n",
       "      <td>...</td>\n",
       "      <td>s</td>\n",
       "      <td>w</td>\n",
       "      <td>w</td>\n",
       "      <td>p</td>\n",
       "      <td>w</td>\n",
       "      <td>o</td>\n",
       "      <td>p</td>\n",
       "      <td>k</td>\n",
       "      <td>s</td>\n",
       "      <td>u</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>e</td>\n",
       "      <td>x</td>\n",
       "      <td>s</td>\n",
       "      <td>g</td>\n",
       "      <td>f</td>\n",
       "      <td>n</td>\n",
       "      <td>f</td>\n",
       "      <td>w</td>\n",
       "      <td>b</td>\n",
       "      <td>k</td>\n",
       "      <td>...</td>\n",
       "      <td>s</td>\n",
       "      <td>w</td>\n",
       "      <td>w</td>\n",
       "      <td>p</td>\n",
       "      <td>w</td>\n",
       "      <td>o</td>\n",
       "      <td>e</td>\n",
       "      <td>n</td>\n",
       "      <td>a</td>\n",
       "      <td>g</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  col0 col1 col2 col3 col4 col5 col6 col7 col8 col9  ... col13 col14 col15  \\\n",
       "0    p    x    s    n    t    p    f    c    n    k  ...     s     w     w   \n",
       "1    e    x    s    y    t    a    f    c    b    k  ...     s     w     w   \n",
       "2    e    b    s    w    t    l    f    c    b    n  ...     s     w     w   \n",
       "3    p    x    y    w    t    p    f    c    n    n  ...     s     w     w   \n",
       "4    e    x    s    g    f    n    f    w    b    k  ...     s     w     w   \n",
       "\n",
       "  col16 col17 col18 col19 col20 col21 col22  \n",
       "0     p     w     o     p     k     s     u  \n",
       "1     p     w     o     p     n     n     g  \n",
       "2     p     w     o     p     n     n     m  \n",
       "3     p     w     o     p     k     s     u  \n",
       "4     p     w     o     e     n     a     g  \n",
       "\n",
       "[5 rows x 23 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_m.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Анализ датафрейма и преобразование столбцов"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Основная информация по датафрейму (учитывает только числовые столбцы):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "По этим статистикам можно понять, одинаковый ли масштаб имеют признаки (например, Age измеряется от 0 до 80, а Pclass в единицах - от 1 до 3). Также можно понять, есть ли пропуски в данных (по count). Например, в графе Age есть пропуски."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Также можно применять операции .sum(), .mean(), .std() к отдельным столбцам датафрейма (они игнорируют пропущенные значения при вычислении статистик):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29.69911764705882"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Age\"].mean() # средний возраст"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "28693.9493"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Fare\"].sum() # суммарная стоимость билетов"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Еще одна часто используемая статистика -- уникальные значения в столбце:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'female', 'male', 'unknown'}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set(data[\"Sex\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Уникальные значения в столбце с числом строк с таким значением:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "male       574\n",
       "female     312\n",
       "unknown      5\n",
       "Name: Sex, dtype: int64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Sex\"].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Эта операция часто используется для анализа частот категорий в категориальных признаках, то есть таких, которые принимают значения из заданного множества (город, образование, пол и т. д.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Часто при обучении моделей машинного обучения полезно создавать новые признаки на основе имеющихся, то есть создавать новые столбцы как функции от существующих. Для этого пригодятся описанные ниже операции: map и apply.\n",
    "\n",
    "Перекодирование столбца с помощью функции map:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Name\n",
       "Braund; Mr. Owen Harris                                1\n",
       "Cumings; Mrs. John Bradley (Florence Briggs Thayer)   -1\n",
       "Heikkinen; Miss. Laina                                -1\n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)          -1\n",
       "Allen; Mr. William Henry                               1\n",
       "Name: Sex, dtype: int64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sex = data[\"Sex\"]\n",
    "sex.map({\"male\":1, \"female\":-1, \"unknown\":0}).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Функция apply: применение функции поэлементно к столбцу или строке (apply возвращает результат и никак не модифицирует датафрейм, поэтому записываем результат в нвоый столбец \"NewAge\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[\"NewAge\"] = data[\"Age\"].apply(lambda x: x+100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>NewAge</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Braund; Mr. Owen Harris</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>122.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cumings; Mrs. John Bradley (Florence Briggs Thayer)</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings; Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>138.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Heikkinen; Miss. Laina</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen; Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>126.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Futrelle; Mrs. Jacques Heath (Lily May Peel)</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle; Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Allen; Mr. William Henry</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen; Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    PassengerId  Survived  \\\n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                                       1         0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...            2         1   \n",
       "Heikkinen; Miss. Laina                                        3         1   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)                  4         1   \n",
       "Allen; Mr. William Henry                                      5         0   \n",
       "\n",
       "                                                    Pclass  \\\n",
       "Name                                                         \n",
       "Braund; Mr. Owen Harris                                  3   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...       1   \n",
       "Heikkinen; Miss. Laina                                   3   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             1   \n",
       "Allen; Mr. William Henry                                 3   \n",
       "\n",
       "                                                                                                 Name  \\\n",
       "Name                                                                                                    \n",
       "Braund; Mr. Owen Harris                                                       Braund; Mr. Owen Harris   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  Cumings; Mrs. John Bradley (Florence Briggs Th...   \n",
       "Heikkinen; Miss. Laina                                                         Heikkinen; Miss. Laina   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             Futrelle; Mrs. Jacques Heath (Lily May Peel)   \n",
       "Allen; Mr. William Henry                                                     Allen; Mr. William Henry   \n",
       "\n",
       "                                                       Sex   Age  SibSp  \\\n",
       "Name                                                                      \n",
       "Braund; Mr. Owen Harris                               male  22.0      1   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  female  38.0      1   \n",
       "Heikkinen; Miss. Laina                              female  26.0      0   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        female  35.0      1   \n",
       "Allen; Mr. William Henry                              male  35.0      0   \n",
       "\n",
       "                                                    Parch            Ticket  \\\n",
       "Name                                                                          \n",
       "Braund; Mr. Owen Harris                                 0         A/5 21171   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...      0          PC 17599   \n",
       "Heikkinen; Miss. Laina                                  0  STON/O2. 3101282   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)            0            113803   \n",
       "Allen; Mr. William Henry                                0            373450   \n",
       "\n",
       "                                                       Fare Cabin Embarked  \\\n",
       "Name                                                                         \n",
       "Braund; Mr. Owen Harris                              7.2500   NaN        S   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  71.2833   C85        C   \n",
       "Heikkinen; Miss. Laina                               7.9250   NaN        S   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        53.1000  C123        S   \n",
       "Allen; Mr. William Henry                             8.0500   NaN        S   \n",
       "\n",
       "                                                    NewAge  \n",
       "Name                                                        \n",
       "Braund; Mr. Owen Harris                              122.0  \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...   138.0  \n",
       "Heikkinen; Miss. Laina                               126.0  \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)         135.0  \n",
       "Allen; Mr. William Henry                             135.0  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Анализ датафрейма обычно включает ответы на вопросы вида \"Сколько объектов (или какие объекты) удовлетворяют условию?\". Для выполнения таких запросов пригодятся операции сравнения, groupby и индексации.\n",
    "\n",
    "Функция groupby: создание групп по значению какого-то столбца (или группы столбцов)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sex\n",
       "female     781.417515\n",
       "male       947.075377\n",
       "unknown    371.589878\n",
       "Name: Age, dtype: float64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.groupby(\"Sex\")['Age'].apply(lambda x: x.mean()**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создаем столбец с фамилией:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[\"Family\"] = data[\"Name\"].apply(lambda s: s.split(\";\")[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>NewAge</th>\n",
       "      <th>Family</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Braund; Mr. Owen Harris</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>122.0</td>\n",
       "      <td>Braund</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         PassengerId  Survived  Pclass  \\\n",
       "Name                                                     \n",
       "Braund; Mr. Owen Harris            1         0       3   \n",
       "\n",
       "                                            Name   Sex   Age  SibSp  Parch  \\\n",
       "Name                                                                         \n",
       "Braund; Mr. Owen Harris  Braund; Mr. Owen Harris  male  22.0      1      0   \n",
       "\n",
       "                            Ticket  Fare Cabin Embarked  NewAge  Family  \n",
       "Name                                                                     \n",
       "Braund; Mr. Owen Harris  A/5 21171  7.25   NaN        S   122.0  Braund  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Сколько семей, в которых больше трех человек (семья - множество людей с одной фамилией Family)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(data.groupby(\"Family\")[\"Age\"].count() > 3).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Family\n",
       "Abbing     1\n",
       "Abbott     2\n",
       "Abelson    2\n",
       "Adahl      1\n",
       "Adams      1\n",
       "Name: Age, dtype: int64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.groupby(\"Family\")[\"Age\"].count().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Выбор строк по условию (выражение в квадратных скобках создает маску из True и False, выбираются все строки с True):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>NewAge</th>\n",
       "      <th>Family</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Braund; Mr. Owen Harris</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>122.0</td>\n",
       "      <td>Braund</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Allen; Mr. William Henry</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen; Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Allen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>McCarthy; Mr. Timothy J</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>McCarthy; Mr. Timothy J</td>\n",
       "      <td>male</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17463</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>E46</td>\n",
       "      <td>S</td>\n",
       "      <td>154.0</td>\n",
       "      <td>McCarthy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Saundercock; Mr. William Henry</th>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Saundercock; Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5. 2151</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>120.0</td>\n",
       "      <td>Saundercock</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fynney; Mr. Joseph J</th>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Fynney; Mr. Joseph J</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>239865</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Fynney</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                PassengerId  Survived  Pclass  \\\n",
       "Name                                                            \n",
       "Braund; Mr. Owen Harris                   1         0       3   \n",
       "Allen; Mr. William Henry                  5         0       3   \n",
       "McCarthy; Mr. Timothy J                   7         0       1   \n",
       "Saundercock; Mr. William Henry           13         0       3   \n",
       "Fynney; Mr. Joseph J                     21         0       2   \n",
       "\n",
       "                                                          Name   Sex   Age  \\\n",
       "Name                                                                         \n",
       "Braund; Mr. Owen Harris                Braund; Mr. Owen Harris  male  22.0   \n",
       "Allen; Mr. William Henry              Allen; Mr. William Henry  male  35.0   \n",
       "McCarthy; Mr. Timothy J                McCarthy; Mr. Timothy J  male  54.0   \n",
       "Saundercock; Mr. William Henry  Saundercock; Mr. William Henry  male  20.0   \n",
       "Fynney; Mr. Joseph J                      Fynney; Mr. Joseph J  male  35.0   \n",
       "\n",
       "                                SibSp  Parch     Ticket     Fare Cabin  \\\n",
       "Name                                                                     \n",
       "Braund; Mr. Owen Harris             1      0  A/5 21171   7.2500   NaN   \n",
       "Allen; Mr. William Henry            0      0     373450   8.0500   NaN   \n",
       "McCarthy; Mr. Timothy J             0      0      17463  51.8625   E46   \n",
       "Saundercock; Mr. William Henry      0      0  A/5. 2151   8.0500   NaN   \n",
       "Fynney; Mr. Joseph J                0      0     239865  26.0000   NaN   \n",
       "\n",
       "                               Embarked  NewAge       Family  \n",
       "Name                                                          \n",
       "Braund; Mr. Owen Harris               S   122.0       Braund  \n",
       "Allen; Mr. William Henry              S   135.0        Allen  \n",
       "McCarthy; Mr. Timothy J               S   154.0     McCarthy  \n",
       "Saundercock; Mr. William Henry        S   120.0  Saundercock  \n",
       "Fynney; Mr. Joseph J                  S   135.0       Fynney  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[(data.Age > 10) & (data.Sex==\"male\")].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "__Задания:__\n",
    "\n",
    "Какова доля семей, в которых минимальный возраст меньше 20 (семьи с детьми)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Family\n",
       "Abbing           42.0\n",
       "Abbott           16.0\n",
       "Abelson          28.0\n",
       "Adahl            30.0\n",
       "Adams            26.0\n",
       "                 ... \n",
       "de Mulder        30.0\n",
       "de Pelsmaeker    16.0\n",
       "del Carlo        29.0\n",
       "van Billiard     40.5\n",
       "van Melkebeke     NaN\n",
       "Name: Age, Length: 667, dtype: float64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# your code here\n",
    "data.groupby(\"Family\")[\"Age\"].min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.18590704647676162"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(data.groupby(\"Family\")[\"Age\"].min()<20).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Какова доля выживших пассажиров из класса 3? А пассажиров из класса 1?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.24236252545824846"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class3 = data[data[\"Pclass\"]==3]\n",
    "class3[\"Survived\"].mean()\n",
    "# your code here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6296296296296297"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class3 = data[data[\"Pclass\"]==1]\n",
    "class3[\"Survived\"].mean()\n",
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Доля выживших пассажиров больше 50%?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3838383838383838"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Survived\"].mean()\n",
    "# your code here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Survived\"].mean()>0.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создайте столбец \"IsChild\", который равен 1, если возраст меньше 20, и 0 иначе. Для пропущенных значений поведение функции может быть произвольным."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>NewAge</th>\n",
       "      <th>Family</th>\n",
       "      <th>IsChild</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Braund; Mr. Owen Harris</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>122.0</td>\n",
       "      <td>Braund</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cumings; Mrs. John Bradley (Florence Briggs Thayer)</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings; Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>138.0</td>\n",
       "      <td>Cumings</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Heikkinen; Miss. Laina</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen; Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>126.0</td>\n",
       "      <td>Heikkinen</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Futrelle; Mrs. Jacques Heath (Lily May Peel)</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle; Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Futrelle</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Allen; Mr. William Henry</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen; Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Allen</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    PassengerId  Survived  \\\n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                                       1         0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...            2         1   \n",
       "Heikkinen; Miss. Laina                                        3         1   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)                  4         1   \n",
       "Allen; Mr. William Henry                                      5         0   \n",
       "\n",
       "                                                    Pclass  \\\n",
       "Name                                                         \n",
       "Braund; Mr. Owen Harris                                  3   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...       1   \n",
       "Heikkinen; Miss. Laina                                   3   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             1   \n",
       "Allen; Mr. William Henry                                 3   \n",
       "\n",
       "                                                                                                 Name  \\\n",
       "Name                                                                                                    \n",
       "Braund; Mr. Owen Harris                                                       Braund; Mr. Owen Harris   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  Cumings; Mrs. John Bradley (Florence Briggs Th...   \n",
       "Heikkinen; Miss. Laina                                                         Heikkinen; Miss. Laina   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             Futrelle; Mrs. Jacques Heath (Lily May Peel)   \n",
       "Allen; Mr. William Henry                                                     Allen; Mr. William Henry   \n",
       "\n",
       "                                                       Sex   Age  SibSp  \\\n",
       "Name                                                                      \n",
       "Braund; Mr. Owen Harris                               male  22.0      1   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  female  38.0      1   \n",
       "Heikkinen; Miss. Laina                              female  26.0      0   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        female  35.0      1   \n",
       "Allen; Mr. William Henry                              male  35.0      0   \n",
       "\n",
       "                                                    Parch            Ticket  \\\n",
       "Name                                                                          \n",
       "Braund; Mr. Owen Harris                                 0         A/5 21171   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...      0          PC 17599   \n",
       "Heikkinen; Miss. Laina                                  0  STON/O2. 3101282   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)            0            113803   \n",
       "Allen; Mr. William Henry                                0            373450   \n",
       "\n",
       "                                                       Fare Cabin Embarked  \\\n",
       "Name                                                                         \n",
       "Braund; Mr. Owen Harris                              7.2500   NaN        S   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  71.2833   C85        C   \n",
       "Heikkinen; Miss. Laina                               7.9250   NaN        S   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        53.1000  C123        S   \n",
       "Allen; Mr. William Henry                             8.0500   NaN        S   \n",
       "\n",
       "                                                    NewAge     Family  IsChild  \n",
       "Name                                                                            \n",
       "Braund; Mr. Owen Harris                              122.0     Braund        0  \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...   138.0    Cumings        0  \n",
       "Heikkinen; Miss. Laina                               126.0  Heikkinen        0  \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)         135.0   Futrelle        0  \n",
       "Allen; Mr. William Henry                             135.0      Allen        0  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# your code here\n",
    "def transform(age):\n",
    "    if age < 20:\n",
    "        return 1\n",
    "    else:\n",
    "        return 0\n",
    "data[\"IsChild\"] = data[\"Age\"].apply(transform)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Рассмотрим еще две операции, которые могут потребоваться для обработки данных: заполнение пропущенных значений и кодирование категориальных признаков."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Для заполнения пропущенных значений используется функция .fillna (в качестве аргумента нужно указать, каким значением нужно заполнить пропуски - для примера заполним нулями):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Name\n",
       "Braund; Mr. Owen Harris                                22.0\n",
       "Cumings; Mrs. John Bradley (Florence Briggs Thayer)    38.0\n",
       "Heikkinen; Miss. Laina                                 26.0\n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)           35.0\n",
       "Allen; Mr. William Henry                               35.0\n",
       "Name: Age, dtype: float64"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[\"Age\"].fillna(0).head() # head нужен только для печати результата"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Результат нужно снова присвоить в data[\"Age\"] (либо указать inplace=True в аргументах fillna)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Категориальные признаки обычно задаются текстом, и их нужно перекодировать в числа. Можно просто пронумеровать категории - выше мы так сделали с признаком Sex. Но если категорий больше двух, такой подхд может привести к нижкому качеству модели. Обычно используется другой подход: вместо одного категориального признака создается серия бинарных признаков (принимают значения 0 и 1), для каждого объекта 1 ставится в признак, соответствующий его категории, в остальные признаки ставится 0. В pandas такая операция называется get_dummies, воспользуемся ей для кодирования Sex: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>NewAge</th>\n",
       "      <th>Family</th>\n",
       "      <th>Sex_female</th>\n",
       "      <th>Sex_male</th>\n",
       "      <th>Sex_unknown</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Braund; Mr. Owen Harris</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund; Mr. Owen Harris</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>122.0</td>\n",
       "      <td>Braund</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cumings; Mrs. John Bradley (Florence Briggs Thayer)</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings; Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>138.0</td>\n",
       "      <td>Cumings</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Heikkinen; Miss. Laina</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen; Miss. Laina</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>126.0</td>\n",
       "      <td>Heikkinen</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Futrelle; Mrs. Jacques Heath (Lily May Peel)</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle; Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Futrelle</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Allen; Mr. William Henry</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen; Mr. William Henry</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Allen</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    PassengerId  Survived  \\\n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                                       1         0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...            2         1   \n",
       "Heikkinen; Miss. Laina                                        3         1   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)                  4         1   \n",
       "Allen; Mr. William Henry                                      5         0   \n",
       "\n",
       "                                                    Pclass  \\\n",
       "Name                                                         \n",
       "Braund; Mr. Owen Harris                                  3   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...       1   \n",
       "Heikkinen; Miss. Laina                                   3   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             1   \n",
       "Allen; Mr. William Henry                                 3   \n",
       "\n",
       "                                                                                                 Name  \\\n",
       "Name                                                                                                    \n",
       "Braund; Mr. Owen Harris                                                       Braund; Mr. Owen Harris   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  Cumings; Mrs. John Bradley (Florence Briggs Th...   \n",
       "Heikkinen; Miss. Laina                                                         Heikkinen; Miss. Laina   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)             Futrelle; Mrs. Jacques Heath (Lily May Peel)   \n",
       "Allen; Mr. William Henry                                                     Allen; Mr. William Henry   \n",
       "\n",
       "                                                     Age  SibSp  Parch  \\\n",
       "Name                                                                     \n",
       "Braund; Mr. Owen Harris                             22.0      1      0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...  38.0      1      0   \n",
       "Heikkinen; Miss. Laina                              26.0      0      0   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        35.0      1      0   \n",
       "Allen; Mr. William Henry                            35.0      0      0   \n",
       "\n",
       "                                                              Ticket     Fare  \\\n",
       "Name                                                                            \n",
       "Braund; Mr. Owen Harris                                    A/5 21171   7.2500   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...          PC 17599  71.2833   \n",
       "Heikkinen; Miss. Laina                              STON/O2. 3101282   7.9250   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)                  113803  53.1000   \n",
       "Allen; Mr. William Henry                                      373450   8.0500   \n",
       "\n",
       "                                                   Cabin Embarked  NewAge  \\\n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                              NaN        S   122.0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...   C85        C   138.0   \n",
       "Heikkinen; Miss. Laina                               NaN        S   126.0   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)        C123        S   135.0   \n",
       "Allen; Mr. William Henry                             NaN        S   135.0   \n",
       "\n",
       "                                                       Family  Sex_female  \\\n",
       "Name                                                                        \n",
       "Braund; Mr. Owen Harris                                Braund           0   \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...    Cumings           1   \n",
       "Heikkinen; Miss. Laina                              Heikkinen           1   \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)         Futrelle           1   \n",
       "Allen; Mr. William Henry                                Allen           0   \n",
       "\n",
       "                                                    Sex_male  Sex_unknown  \n",
       "Name                                                                       \n",
       "Braund; Mr. Owen Harris                                    1            0  \n",
       "Cumings; Mrs. John Bradley (Florence Briggs Tha...         0            0  \n",
       "Heikkinen; Miss. Laina                                     0            0  \n",
       "Futrelle; Mrs. Jacques Heath (Lily May Peel)               0            0  \n",
       "Allen; Mr. William Henry                                   1            0  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(data, columns=[\"Sex\"]).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В правой части появились новые столбцы. Вновь можно указать inplace=True в качестве аргумента get_dummies, чтобы не нужно было пересохранять результат в переменную."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Преобразование данных\n",
    "Для обучения моделей машинного обучения нужно, чтобы все данные были числовые. В данных Титаника есть нечисловые столбцы, например строковые. Ваша задача - преобразовать данные в полностью числовой вид. Для этого Вам понадобится выделить числовые признаки из строковых. Выполните следующие операции с данными:\n",
    "* Закодируйте признак Sex любым из двух способов, описанных выше, и добавьте в датафрейм преобразованный(е) признак(и), сам признак Sex удалите.\n",
    "* Проверьте, что признак Embarked принимает значения из небольшого множества (воспользуйтесь set или value_counts). Закодируйте его с помощью get_dummies, сам признак Embarked удалите.\n",
    "* Добавьте признак \"длина номера билета\" (в символах), вам потребуется команда apply. Признак Ticket удалите.\n",
    "* Добавьте признак CabinClass, который вычисляется по признаку Class взятием первой буквы из строки. Полученный признак CabinClass закодируйте с помощью get_dummies. Признак Class удалите.\n",
    "* Сколько различных значений принимает признак Family? Удалите его, а также признак Name.\n",
    "* Удалите признаки PassengerId (id обычно не информативны), NewAge (мы его добавляли для иллюстрации команды apply).\n",
    "* Теперь у вас должны остаться только числовые признаки, но в них могут быть пропуски. Воспользуйтесь командой data.describe() или data.count(), чтобы понять, в каких признаках есть пропуски. Заполните пропуски в столбце Age и других числовым стобцах, в которых есть пропуски, средним значением по соответствующему столбцу.\n",
    "\n",
    "После выполнения всех шагов вы получите набор данных, который можно использовать для построения модели : )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "py37_default",
   "language": "python",
   "name": "conda-env-py37_default-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}