GorczycaTitanicCleaning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Czyszczenie danych i przygotowanie zbioru do analizy\n",
    "\n",
    "Pracę można wykonać w R lub Python (rekomendowany Python+pandas/numpy)\n",
    "\n",
    "Pobierz zbiór danych TitanicMess.tsv. Jest to plik którego separatorem jest tabulacja\n",
    "Przygotuj skoroszyt Jupyter - w którym będziesz dokumentować poszczególne kroki analizy a następnie czyszczenia zbioru\n",
    "Zapoznaj się ze zbiorem - przyglądnij mu się. Zidentyfikuj i opisz problemy z jakością danych które znalazłeś (aś) w poszczególnych kolumnach, w całym zbiorze. Opisz jakie problemy zidentyfikowałas(eś) i pokaz gdzie je widać albo na podstawie czego o nich wnioskujesz.\n",
    "Wyczyść zbiór, tzn usuń problemy które znalazłeś/aś. Opisz krótko co i dlaczego robisz - i jaki efekt masz nadzieję osiągnąć. No i .. pokaż że się udało, bo problemu już nie ma.\n",
    "Zapisz oczyszczony zbiór pod nazwą TitanicCleaned.tsv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Do rozpoczęcia pracy nad zbiorem wymagane jest zaimportowanie bibliotek podstawowych do tworzenia funkcjnalności.<br>\n",
    "W tym sprawozdaniu wykorzystano biblioteki numpy, pandas i math.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'en_US.UTF-8'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import math\n",
    "import locale\n",
    "locale.setlocale( locale.LC_ALL, 'en_US.UTF-8' )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejny etap to wczytanie zbioru i sprawdzenie poprawności importu.<br>\n",
    "Ustawiono **znak tabulacji** jako separator.<br>\n",
    "Ustawiono **PassengerID** jako kolumnę indeksującą."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>ship</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7,25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>Titanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71,2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>Titanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7,925</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>Titanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53,1</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>Titanic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8,05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>Titanic</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass  \\\n",
       "PassengerId                     \n",
       "1                   0       3   \n",
       "2                   1       1   \n",
       "3                   1       3   \n",
       "4                   1       1   \n",
       "5                   0       3   \n",
       "\n",
       "                                                          Name     Sex Age  \\\n",
       "PassengerId                                                                  \n",
       "1                                      Braund, Mr. Owen Harris    male  22   \n",
       "2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38   \n",
       "3                                       Heikkinen, Miss. Laina  female  26   \n",
       "4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35   \n",
       "5                                     Allen, Mr. William Henry    male  35   \n",
       "\n",
       "             SibSp  Parch            Ticket     Fare Cabin Embarked     ship  \n",
       "PassengerId                                                                   \n",
       "1                1      0         A/5 21171     7,25   NaN        S  Titanic  \n",
       "2                1      0          PC 17599  71,2833   C85        C  Titanic  \n",
       "3                0      0  STON/O2. 3101282    7,925   NaN        S  Titanic  \n",
       "4                1      0            113803     53,1  C123        S  Titanic  \n",
       "5                0      0            373450     8,05   NaN        S  Titanic  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_df = pd.read_csv('TitanicMess.tsv', sep='\\t', index_col='PassengerId', decimal=',')\n",
    "titanic_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejny etap to zidentyfikowanie kolumn z błędnymi lub pustymi wartościami."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived     int64\n",
       "Pclass       int64\n",
       "Name        object\n",
       "Sex         object\n",
       "Age         object\n",
       "SibSp        int64\n",
       "Parch        int64\n",
       "Ticket      object\n",
       "Fare        object\n",
       "Cabin       object\n",
       "Embarked    object\n",
       "ship        object\n",
       "dtype: object"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wynik powyższego bloku kodu pozwala na wywnioskowanie, że prawdopodobnie kolumny Age i Fare zawierają błędne dane, gdyż nazwa sugeruje wartości liczbowe, a typ jest oznaczony jako object.<br>\n",
    "Teraz sprawdzona zostanie ilość wartości pustych."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived      0\n",
       "Pclass        0\n",
       "Name          0\n",
       "Sex           0\n",
       "Age         173\n",
       "SibSp         0\n",
       "Parch         0\n",
       "Ticket        0\n",
       "Fare          0\n",
       "Cabin       685\n",
       "Embarked      2\n",
       "ship          0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolumny Age, Cabin oraz Embarked zawierają puste wartości."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Przeglądając zbiór danych można zauważyć, że wszystkie rekordy w kolumnie Ship zawierają wartość *Titanic*. <br>\n",
    "Pozwala to na usunięcie kolumny w celu optymalizacji, gdyż cały zbiór dotyczy tylko statku Titanic, więc jest ona zwyczajnie zbędna."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7,25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71,2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7,925</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53,1</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8,05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass  \\\n",
       "PassengerId                     \n",
       "1                   0       3   \n",
       "2                   1       1   \n",
       "3                   1       3   \n",
       "4                   1       1   \n",
       "5                   0       3   \n",
       "\n",
       "                                                          Name     Sex Age  \\\n",
       "PassengerId                                                                  \n",
       "1                                      Braund, Mr. Owen Harris    male  22   \n",
       "2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38   \n",
       "3                                       Heikkinen, Miss. Laina  female  26   \n",
       "4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35   \n",
       "5                                     Allen, Mr. William Henry    male  35   \n",
       "\n",
       "             SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "PassengerId                                                          \n",
       "1                1      0         A/5 21171     7,25   NaN        S  \n",
       "2                1      0          PC 17599  71,2833   C85        C  \n",
       "3                0      0  STON/O2. 3101282    7,925   NaN        S  \n",
       "4                1      0            113803     53,1  C123        S  \n",
       "5                0      0            373450     8,05   NaN        S  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_clear_columns = titanic_df.drop(columns='ship')\n",
    "titanic_clear_columns.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Głębsza analiza zbioru danych pozwala na wysunięcie kolejnych wniosków:<br>\n",
    "- **Survived** - cecha wyjściowa, gdzie 1 oznacza, że pasażer przeżył katastrofę, a 0 - nie\n",
    "- **Pclass** - klasa pasażera (1,2,3)\n",
    "- **Age** - wiek pasażera"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wywnioskowano z wcześniejszej analizy, że kolumna **Age** zawiera puste wartości.<br>\n",
    "Postanowiono wypełnić je wartościami średnimi z całej kolumny, co umożliwi biblioteka PANDAS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "can only concatenate str (not \"int\") to str",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-6-c711e433e4cb>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mtitanic_clear_columns\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Age'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32mC:\\CollegePrograms\\Anaconda3\\lib\\site-packages\\pandas\\core\\generic.py\u001b[0m in \u001b[0;36mstat_func\u001b[1;34m(self, axis, skipna, level, numeric_only, **kwargs)\u001b[0m\n\u001b[0;32m  11466\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mlevel\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m  11467\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_agg_by_level\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mskipna\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mskipna\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m> 11468\u001b[1;33m         return self._reduce(\n\u001b[0m\u001b[0;32m  11469\u001b[0m             \u001b[0mfunc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mname\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mskipna\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mskipna\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnumeric_only\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mnumeric_only\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m  11470\u001b[0m         )\n",
      "\u001b[1;32mC:\\CollegePrograms\\Anaconda3\\lib\\site-packages\\pandas\\core\\series.py\u001b[0m in \u001b[0;36m_reduce\u001b[1;34m(self, op, name, axis, skipna, numeric_only, filter_type, **kwds)\u001b[0m\n\u001b[0;32m   4234\u001b[0m                 )\n\u001b[0;32m   4235\u001b[0m             \u001b[1;32mwith\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0merrstate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mall\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m\"ignore\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 4236\u001b[1;33m                 \u001b[1;32mreturn\u001b[0m \u001b[0mop\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdelegate\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mskipna\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mskipna\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   4237\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   4238\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0m_reindex_indexer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnew_index\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mindexer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcopy\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\CollegePrograms\\Anaconda3\\lib\\site-packages\\pandas\\core\\nanops.py\u001b[0m in \u001b[0;36m_f\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m     69\u001b[0m             \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     70\u001b[0m                 \u001b[1;32mwith\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0merrstate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0minvalid\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m\"ignore\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 71\u001b[1;33m                     \u001b[1;32mreturn\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     72\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mValueError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     73\u001b[0m                 \u001b[1;31m# we want to transform an object array\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\CollegePrograms\\Anaconda3\\lib\\site-packages\\pandas\\core\\nanops.py\u001b[0m in \u001b[0;36mf\u001b[1;34m(values, axis, skipna, **kwds)\u001b[0m\n\u001b[0;32m    127\u001b[0m                     \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0malt\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mskipna\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mskipna\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    128\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 129\u001b[1;33m                 \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0malt\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mskipna\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mskipna\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    130\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    131\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\CollegePrograms\\Anaconda3\\lib\\site-packages\\pandas\\core\\nanops.py\u001b[0m in \u001b[0;36mnanmean\u001b[1;34m(values, axis, skipna, mask)\u001b[0m\n\u001b[0;32m    561\u001b[0m         \u001b[0mdtype_count\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    562\u001b[0m     \u001b[0mcount\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_get_counts\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmask\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdtype_count\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 563\u001b[1;33m     \u001b[0mthe_sum\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0m_ensure_numeric\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msum\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mdtype_sum\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    564\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    565\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0maxis\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m \u001b[1;32mand\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mthe_sum\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"ndim\"\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mC:\\CollegePrograms\\Anaconda3\\lib\\site-packages\\numpy\\core\\_methods.py\u001b[0m in \u001b[0;36m_sum\u001b[1;34m(a, axis, dtype, out, keepdims, initial, where)\u001b[0m\n\u001b[0;32m     45\u001b[0m def _sum(a, axis=None, dtype=None, out=None, keepdims=False,\n\u001b[0;32m     46\u001b[0m          initial=_NoValue, where=True):\n\u001b[1;32m---> 47\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0mumr_sum\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mout\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkeepdims\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minitial\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mwhere\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     48\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     49\u001b[0m def _prod(a, axis=None, dtype=None, out=None, keepdims=False,\n",
      "\u001b[1;31mTypeError\u001b[0m: can only concatenate str (not \"int\") to str"
     ]
    }
   ],
   "source": [
    "titanic_clear_columns['Age'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wywołanie metody wyciągającej średnią (mean()) powoduje błąd. Spowodowane jest to faktem, że kolumna zawiera wartości innego typu niż numeryczne.<br>\n",
    "Należy znaleźć błędne rekordy poprzez blok try-catch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wrong age value for passegnerId 58\n",
      "Wrong age value for passegnerId 79\n",
      "Wrong age value for passegnerId 112\n",
      "Wrong age value for passegnerId 117\n",
      "Wrong age value for passegnerId 123\n",
      "Wrong age value for passegnerId 124\n",
      "Wrong age value for passegnerId 149\n",
      "Wrong age value for passegnerId 153\n",
      "Wrong age value for passegnerId 154\n",
      "Wrong age value for passegnerId 204\n",
      "Wrong age value for passegnerId 228\n",
      "Wrong age value for passegnerId 297\n",
      "Wrong age value for passegnerId 306\n",
      "Wrong age value for passegnerId 332\n",
      "Wrong age value for passegnerId 470\n",
      "Wrong age value for passegnerId 526\n",
      "Wrong age value for passegnerId 645\n",
      "Wrong age value for passegnerId 677\n",
      "Wrong age value for passegnerId 736\n",
      "Wrong age value for passegnerId 756\n",
      "Wrong age value for passegnerId 768\n",
      "Wrong age value for passegnerId 804\n",
      "Wrong age value for passegnerId 815\n",
      "Wrong age value for passegnerId 832\n",
      "Wrong age value for passegnerId 844\n",
      "Number of not numerical age values 25\n"
     ]
    }
   ],
   "source": [
    "wrongAges=0 ## number of wrong fields\n",
    "for index, row in titanic_clear_columns.iterrows():\n",
    "    try:\n",
    "        float(row['Age'])  ## parsing to float\n",
    "    except ValueError:\n",
    "        wrongAges = wrongAges + 1\n",
    "        print(\"Wrong age value for passegnerId \" + str(index))\n",
    "    \n",
    "print(\"Number of not numerical age values \" + str(wrongAges)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Należy przekonwertować powyższe wartości do typu float lub jeśli niemożliwe, to ustawienie wartości NaN.<br>\n",
    "Zakres wieku ustawiono na 0-123 (123 to wiek najdłużej żyjącego człowieka w historii).<br>\n",
    "Po wykonaniu powyższych instrukcji nastąpiło sprawdzenie typu kolumn,  w celu potwierdzenia poprawności operacji."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.0\n",
      "38.0\n",
      "26.0\n",
      "35.0\n",
      "35.0\n",
      "nan\n",
      "54.0\n",
      "2.0\n",
      "27.0\n",
      "14.0\n",
      "4.0\n",
      "58.0\n",
      "20.0\n",
      "4.0\n",
      "14.0\n",
      "55.0\n",
      "2.0\n",
      "nan\n",
      "31.0\n",
      "nan\n",
      "35.0\n",
      "34.0\n",
      "15.0\n",
      "4.0\n",
      "8.0\n",
      "38.0\n",
      "nan\n",
      "19.0\n",
      "nan\n",
      "nan\n",
      "40.0\n",
      "0.9\n",
      "nan\n",
      "66.0\n",
      "28.0\n",
      "42.0\n",
      "nan\n",
      "21.0\n",
      "18.0\n",
      "14.0\n",
      "40.0\n",
      "27.0\n",
      "nan\n",
      "3.0\n",
      "19.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "18.0\n",
      "7.0\n",
      "21.0\n",
      "49.0\n",
      "29.0\n",
      "65.0\n",
      "nan\n",
      "21.0\n",
      "nan\n",
      "5.0\n",
      "11.0\n",
      "22.0\n",
      "38.0\n",
      "45.0\n",
      "4.0\n",
      "nan\n",
      "nan\n",
      "29.0\n",
      "19.0\n",
      "17.0\n",
      "26.0\n",
      "32.0\n",
      "16.0\n",
      "21.0\n",
      "26.0\n",
      "32.0\n",
      "25.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "30.0\n",
      "22.0\n",
      "29.0\n",
      "nan\n",
      "28.0\n",
      "17.0\n",
      "33.0\n",
      "16.0\n",
      "nan\n",
      "23.0\n",
      "24.0\n",
      "29.0\n",
      "20.0\n",
      "46.0\n",
      "26.0\n",
      "59.0\n",
      "nan\n",
      "71.0\n",
      "23.0\n",
      "34.0\n",
      "34.0\n",
      "28.0\n",
      "nan\n",
      "21.0\n",
      "33.0\n",
      "37.0\n",
      "28.0\n",
      "21.0\n",
      "nan\n",
      "38.0\n",
      "nan\n",
      "47.0\n",
      "nan\n",
      "22.0\n",
      "20.0\n",
      "17.0\n",
      "21.0\n",
      "nan\n",
      "29.0\n",
      "24.0\n",
      "2.0\n",
      "21.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "54.0\n",
      "12.0\n",
      "nan\n",
      "24.0\n",
      "nan\n",
      "45.0\n",
      "33.0\n",
      "20.0\n",
      "47.0\n",
      "29.0\n",
      "25.0\n",
      "23.0\n",
      "19.0\n",
      "37.0\n",
      "16.0\n",
      "24.0\n",
      "nan\n",
      "22.0\n",
      "24.0\n",
      "19.0\n",
      "18.0\n",
      "19.0\n",
      "27.0\n",
      "9.0\n",
      "nan\n",
      "42.0\n",
      "51.0\n",
      "22.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "51.0\n",
      "16.0\n",
      "30.0\n",
      "nan\n",
      "nan\n",
      "44.0\n",
      "40.0\n",
      "26.0\n",
      "17.0\n",
      "1.0\n",
      "9.0\n",
      "nan\n",
      "45.0\n",
      "nan\n",
      "28.0\n",
      "61.0\n",
      "4.0\n",
      "1.0\n",
      "21.0\n",
      "56.0\n",
      "18.0\n",
      "nan\n",
      "50.0\n",
      "30.0\n",
      "36.0\n",
      "nan\n",
      "nan\n",
      "9.0\n",
      "1.0\n",
      "4.0\n",
      "nan\n",
      "nan\n",
      "45.0\n",
      "40.0\n",
      "36.0\n",
      "32.0\n",
      "19.0\n",
      "19.0\n",
      "3.0\n",
      "44.0\n",
      "58.0\n",
      "nan\n",
      "42.0\n",
      "nan\n",
      "24.0\n",
      "28.0\n",
      "nan\n",
      "34.0\n",
      "nan\n",
      "18.0\n",
      "2.0\n",
      "32.0\n",
      "26.0\n",
      "16.0\n",
      "40.0\n",
      "24.0\n",
      "35.0\n",
      "22.0\n",
      "30.0\n",
      "nan\n",
      "31.0\n",
      "27.0\n",
      "42.0\n",
      "32.0\n",
      "30.0\n",
      "16.0\n",
      "27.0\n",
      "51.0\n",
      "nan\n",
      "38.0\n",
      "22.0\n",
      "19.0\n",
      "nan\n",
      "18.0\n",
      "nan\n",
      "35.0\n",
      "29.0\n",
      "59.0\n",
      "5.0\n",
      "24.0\n",
      "nan\n",
      "44.0\n",
      "8.0\n",
      "19.0\n",
      "33.0\n",
      "nan\n",
      "nan\n",
      "29.0\n",
      "22.0\n",
      "30.0\n",
      "44.0\n",
      "25.0\n",
      "24.0\n",
      "37.0\n",
      "54.0\n",
      "nan\n",
      "29.0\n",
      "62.0\n",
      "30.0\n",
      "41.0\n",
      "29.0\n",
      "nan\n",
      "30.0\n",
      "35.0\n",
      "50.0\n",
      "nan\n",
      "3.0\n",
      "52.0\n",
      "40.0\n",
      "nan\n",
      "36.0\n",
      "16.0\n",
      "25.0\n",
      "58.0\n",
      "35.0\n",
      "nan\n",
      "25.0\n",
      "41.0\n",
      "37.0\n",
      "nan\n",
      "63.0\n",
      "45.0\n",
      "nan\n",
      "7.0\n",
      "35.0\n",
      "65.0\n",
      "28.0\n",
      "16.0\n",
      "19.0\n",
      "nan\n",
      "33.0\n",
      "30.0\n",
      "22.0\n",
      "42.0\n",
      "22.0\n",
      "26.0\n",
      "19.0\n",
      "36.0\n",
      "24.0\n",
      "24.0\n",
      "nan\n",
      "nan\n",
      "2.0\n",
      "nan\n",
      "50.0\n",
      "nan\n",
      "nan\n",
      "19.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "17.0\n",
      "30.0\n",
      "30.0\n",
      "24.0\n",
      "18.0\n",
      "26.0\n",
      "28.0\n",
      "43.0\n",
      "26.0\n",
      "24.0\n",
      "54.0\n",
      "31.0\n",
      "40.0\n",
      "22.0\n",
      "27.0\n",
      "30.0\n",
      "22.0\n",
      "nan\n",
      "36.0\n",
      "61.0\n",
      "36.0\n",
      "31.0\n",
      "16.0\n",
      "nan\n",
      "nan\n",
      "38.0\n",
      "16.0\n",
      "nan\n",
      "nan\n",
      "29.0\n",
      "41.0\n",
      "45.0\n",
      "45.0\n",
      "2.0\n",
      "24.0\n",
      "28.0\n",
      "25.0\n",
      "36.0\n",
      "24.0\n",
      "40.0\n",
      "nan\n",
      "3.0\n",
      "42.0\n",
      "23.0\n",
      "nan\n",
      "15.0\n",
      "25.0\n",
      "nan\n",
      "28.0\n",
      "22.0\n",
      "38.0\n",
      "nan\n",
      "nan\n",
      "40.0\n",
      "29.0\n",
      "45.0\n",
      "35.0\n",
      "nan\n",
      "30.0\n",
      "60.0\n",
      "nan\n",
      "nan\n",
      "24.0\n",
      "25.0\n",
      "18.0\n",
      "19.0\n",
      "22.0\n",
      "3.0\n",
      "nan\n",
      "22.0\n",
      "27.0\n",
      "20.0\n",
      "19.0\n",
      "42.0\n",
      "1.0\n",
      "32.0\n",
      "35.0\n",
      "nan\n",
      "18.0\n",
      "1.0\n",
      "36.0\n",
      "nan\n",
      "17.0\n",
      "36.0\n",
      "21.0\n",
      "28.0\n",
      "23.0\n",
      "24.0\n",
      "22.0\n",
      "31.0\n",
      "46.0\n",
      "23.0\n",
      "28.0\n",
      "39.0\n",
      "26.0\n",
      "21.0\n",
      "28.0\n",
      "20.0\n",
      "34.0\n",
      "51.0\n",
      "3.0\n",
      "21.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "33.0\n",
      "nan\n",
      "44.0\n",
      "nan\n",
      "34.0\n",
      "18.0\n",
      "30.0\n",
      "10.0\n",
      "nan\n",
      "21.0\n",
      "29.0\n",
      "28.0\n",
      "18.0\n",
      "nan\n",
      "28.0\n",
      "19.0\n",
      "nan\n",
      "32.0\n",
      "28.0\n",
      "nan\n",
      "42.0\n",
      "17.0\n",
      "50.0\n",
      "14.0\n",
      "21.0\n",
      "24.0\n",
      "64.0\n",
      "31.0\n",
      "45.0\n",
      "20.0\n",
      "25.0\n",
      "28.0\n",
      "nan\n",
      "4.0\n",
      "13.0\n",
      "34.0\n",
      "5.0\n",
      "52.0\n",
      "36.0\n",
      "nan\n",
      "30.0\n",
      "49.0\n",
      "nan\n",
      "29.0\n",
      "65.0\n",
      "nan\n",
      "50.0\n",
      "nan\n",
      "48.0\n",
      "34.0\n",
      "47.0\n",
      "48.0\n",
      "nan\n",
      "38.0\n",
      "nan\n",
      "56.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "38.0\n",
      "33.0\n",
      "23.0\n",
      "22.0\n",
      "nan\n",
      "34.0\n",
      "29.0\n",
      "22.0\n",
      "2.0\n",
      "9.0\n",
      "nan\n",
      "50.0\n",
      "63.0\n",
      "25.0\n",
      "nan\n",
      "35.0\n",
      "58.0\n",
      "30.0\n",
      "9.0\n",
      "nan\n",
      "21.0\n",
      "55.0\n",
      "71.0\n",
      "21.0\n",
      "nan\n",
      "54.0\n",
      "nan\n",
      "25.0\n",
      "24.0\n",
      "17.0\n",
      "21.0\n",
      "nan\n",
      "37.0\n",
      "16.0\n",
      "18.0\n",
      "33.0\n",
      "nan\n",
      "28.0\n",
      "26.0\n",
      "29.0\n",
      "nan\n",
      "36.0\n",
      "54.0\n",
      "24.0\n",
      "47.0\n",
      "34.0\n",
      "nan\n",
      "36.0\n",
      "32.0\n",
      "38.0\n",
      "22.0\n",
      "nan\n",
      "44.0\n",
      "nan\n",
      "nan\n",
      "50.0\n",
      "nan\n",
      "39.0\n",
      "23.0\n",
      "2.0\n",
      "nan\n",
      "17.0\n",
      "nan\n",
      "30.0\n",
      "7.0\n",
      "45.0\n",
      "30.0\n",
      "nan\n",
      "22.0\n",
      "36.0\n",
      "9.0\n",
      "11.0\n",
      "32.0\n",
      "50.0\n",
      "64.0\n",
      "19.0\n",
      "nan\n",
      "33.0\n",
      "8.0\n",
      "17.0\n",
      "27.0\n",
      "nan\n",
      "22.0\n",
      "22.0\n",
      "62.0\n",
      "48.0\n",
      "nan\n",
      "39.0\n",
      "36.0\n",
      "nan\n",
      "40.0\n",
      "28.0\n",
      "nan\n",
      "nan\n",
      "24.0\n",
      "19.0\n",
      "29.0\n",
      "nan\n",
      "32.0\n",
      "62.0\n",
      "53.0\n",
      "36.0\n",
      "nan\n",
      "16.0\n",
      "19.0\n",
      "34.0\n",
      "39.0\n",
      "nan\n",
      "32.0\n",
      "25.0\n",
      "39.0\n",
      "54.0\n",
      "36.0\n",
      "nan\n",
      "18.0\n",
      "47.0\n",
      "60.0\n",
      "22.0\n",
      "nan\n",
      "35.0\n",
      "52.0\n",
      "47.0\n",
      "nan\n",
      "37.0\n",
      "36.0\n",
      "nan\n",
      "49.0\n",
      "nan\n",
      "49.0\n",
      "24.0\n",
      "nan\n",
      "nan\n",
      "44.0\n",
      "35.0\n",
      "36.0\n",
      "30.0\n",
      "27.0\n",
      "22.0\n",
      "40.0\n",
      "39.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "35.0\n",
      "24.0\n",
      "34.0\n",
      "26.0\n",
      "4.0\n",
      "26.0\n",
      "27.0\n",
      "42.0\n",
      "20.0\n",
      "21.0\n",
      "21.0\n",
      "61.0\n",
      "57.0\n",
      "21.0\n",
      "26.0\n",
      "nan\n",
      "80.0\n",
      "51.0\n",
      "32.0\n",
      "nan\n",
      "9.0\n",
      "28.0\n",
      "32.0\n",
      "31.0\n",
      "41.0\n",
      "nan\n",
      "20.0\n",
      "24.0\n",
      "2.0\n",
      "nan\n",
      "nan\n",
      "48.0\n",
      "19.0\n",
      "56.0\n",
      "nan\n",
      "23.0\n",
      "nan\n",
      "18.0\n",
      "21.0\n",
      "nan\n",
      "18.0\n",
      "24.0\n",
      "nan\n",
      "32.0\n",
      "23.0\n",
      "58.0\n",
      "50.0\n",
      "40.0\n",
      "47.0\n",
      "36.0\n",
      "20.0\n",
      "32.0\n",
      "nan\n",
      "nan\n",
      "43.0\n",
      "nan\n",
      "40.0\n",
      "31.0\n",
      "70.0\n",
      "31.0\n",
      "nan\n",
      "18.0\n",
      "nan\n",
      "18.0\n",
      "38.0\n",
      "36.0\n",
      "nan\n",
      "27.0\n",
      "20.0\n",
      "14.0\n",
      "60.0\n",
      "25.0\n",
      "14.0\n",
      "19.0\n",
      "18.0\n",
      "15.0\n",
      "31.0\n",
      "4.0\n",
      "nan\n",
      "25.0\n",
      "60.0\n",
      "52.0\n",
      "44.0\n",
      "nan\n",
      "49.0\n",
      "42.0\n",
      "18.0\n",
      "35.0\n",
      "18.0\n",
      "25.0\n",
      "26.0\n",
      "39.0\n",
      "45.0\n",
      "42.0\n",
      "22.0\n",
      "nan\n",
      "24.0\n",
      "nan\n",
      "48.0\n",
      "29.0\n",
      "52.0\n",
      "19.0\n",
      "38.0\n",
      "27.0\n",
      "nan\n",
      "33.0\n",
      "6.0\n",
      "17.0\n",
      "34.0\n",
      "50.0\n",
      "27.0\n",
      "20.0\n",
      "30.0\n",
      "nan\n",
      "25.0\n",
      "25.0\n",
      "29.0\n",
      "11.0\n",
      "nan\n",
      "23.0\n",
      "23.0\n",
      "nan\n",
      "48.0\n",
      "35.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "36.0\n",
      "21.0\n",
      "24.0\n",
      "31.0\n",
      "70.0\n",
      "16.0\n",
      "30.0\n",
      "19.0\n",
      "31.0\n",
      "4.0\n",
      "6.0\n",
      "33.0\n",
      "23.0\n",
      "48.0\n",
      "nan\n",
      "28.0\n",
      "18.0\n",
      "34.0\n",
      "33.0\n",
      "nan\n",
      "41.0\n",
      "20.0\n",
      "36.0\n",
      "16.0\n",
      "51.0\n",
      "nan\n",
      "nan\n",
      "nan\n",
      "32.0\n",
      "24.0\n",
      "48.0\n",
      "57.0\n",
      "nan\n",
      "54.0\n",
      "18.0\n",
      "nan\n",
      "5.0\n",
      "nan\n",
      "43.0\n",
      "13.0\n",
      "17.0\n",
      "29.0\n",
      "nan\n",
      "25.0\n",
      "25.0\n",
      "18.0\n",
      "8.0\n",
      "1.0\n",
      "46.0\n",
      "nan\n",
      "16.0\n",
      "nan\n",
      "nan\n",
      "25.0\n",
      "39.0\n",
      "49.0\n",
      "31.0\n",
      "30.0\n",
      "30.0\n",
      "34.0\n",
      "31.0\n",
      "11.0\n",
      "nan\n",
      "27.0\n",
      "31.0\n",
      "39.0\n",
      "18.0\n",
      "39.0\n",
      "33.0\n",
      "26.0\n",
      "39.0\n",
      "35.0\n",
      "6.0\n",
      "nan\n",
      "nan\n",
      "23.0\n",
      "31.0\n",
      "43.0\n",
      "10.0\n",
      "52.0\n",
      "27.0\n",
      "38.0\n",
      "27.0\n",
      "2.0\n",
      "nan\n",
      "nan\n",
      "1.0\n",
      "nan\n",
      "62.0\n",
      "15.0\n",
      "nan\n",
      "nan\n",
      "23.0\n",
      "18.0\n",
      "39.0\n",
      "21.0\n",
      "nan\n",
      "32.0\n",
      "nan\n",
      "20.0\n",
      "16.0\n",
      "30.0\n",
      "nan\n",
      "17.0\n",
      "42.0\n",
      "nan\n",
      "35.0\n",
      "28.0\n",
      "nan\n",
      "4.0\n",
      "74.0\n",
      "9.0\n",
      "16.0\n",
      "44.0\n",
      "18.0\n",
      "45.0\n",
      "51.0\n",
      "24.0\n",
      "nan\n",
      "41.0\n",
      "21.0\n",
      "48.0\n",
      "nan\n",
      "24.0\n",
      "42.0\n",
      "27.0\n",
      "31.0\n",
      "nan\n",
      "4.0\n",
      "26.0\n",
      "47.0\n",
      "33.0\n",
      "47.0\n",
      "28.0\n",
      "15.0\n",
      "20.0\n",
      "19.0\n",
      "nan\n",
      "56.0\n",
      "25.0\n",
      "33.0\n",
      "22.0\n",
      "28.0\n",
      "25.0\n",
      "39.0\n",
      "27.0\n",
      "19.0\n",
      "nan\n",
      "26.0\n",
      "32.0\n",
      "38.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Survived      int64\n",
       "Pclass        int64\n",
       "Name         object\n",
       "Sex          object\n",
       "Age         float64\n",
       "SibSp         int64\n",
       "Parch         int64\n",
       "Ticket       object\n",
       "Fare         object\n",
       "Cabin        object\n",
       "Embarked     object\n",
       "dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for i, row in titanic_clear_columns.iterrows():\n",
    "    try:\n",
    "        if float(row[\"Age\"]) < 0 or float(row[\"Age\"]) > 123:\n",
    "            titanic_clear_columns.at[i, \"Age\"] = np.NaN\n",
    "        else:\n",
    "            titanic_clear_columns.at[i, \"Age\"] = float(row[\"Age\"])\n",
    "        \n",
    "    except ValueError:\n",
    "        titanic_clear_columns.at[i, \"Age\"] = np.NaN\n",
    "for index, row in titanic_clear_columns.iterrows():\n",
    "    print(row[\"Age\"])\n",
    "\n",
    "titanic_clear_columns[[\"Age\"]] = titanic_clear_columns[[\"Age\"]].astype(float).round(1) ## parsing to float\n",
    "titanic_clear_columns.dtypes ## validate data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Powyższy wynik przedstawia, że operacja zakończyła się sukcesem, co pozwala na wyciągnięcie i zaokrąglenie średniego wieku pasażera."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "round(titanic_clear_columns['Age'].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obliczoną wartość wstawiono do pól pustych i błędnych."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.0\n",
      "38.0\n",
      "26.0\n",
      "35.0\n",
      "35.0\n",
      "30.0\n",
      "54.0\n",
      "2.0\n",
      "27.0\n",
      "14.0\n",
      "4.0\n",
      "58.0\n",
      "20.0\n",
      "4.0\n",
      "14.0\n",
      "55.0\n",
      "2.0\n",
      "30.0\n",
      "31.0\n",
      "30.0\n",
      "35.0\n",
      "34.0\n",
      "15.0\n",
      "4.0\n",
      "8.0\n",
      "38.0\n",
      "30.0\n",
      "19.0\n",
      "30.0\n",
      "30.0\n",
      "40.0\n",
      "0.9\n",
      "30.0\n",
      "66.0\n",
      "28.0\n",
      "42.0\n",
      "30.0\n",
      "21.0\n",
      "18.0\n",
      "14.0\n",
      "40.0\n",
      "27.0\n",
      "30.0\n",
      "3.0\n",
      "19.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "18.0\n",
      "7.0\n",
      "21.0\n",
      "49.0\n",
      "29.0\n",
      "65.0\n",
      "30.0\n",
      "21.0\n",
      "30.0\n",
      "5.0\n",
      "11.0\n",
      "22.0\n",
      "38.0\n",
      "45.0\n",
      "4.0\n",
      "30.0\n",
      "30.0\n",
      "29.0\n",
      "19.0\n",
      "17.0\n",
      "26.0\n",
      "32.0\n",
      "16.0\n",
      "21.0\n",
      "26.0\n",
      "32.0\n",
      "25.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "22.0\n",
      "29.0\n",
      "30.0\n",
      "28.0\n",
      "17.0\n",
      "33.0\n",
      "16.0\n",
      "30.0\n",
      "23.0\n",
      "24.0\n",
      "29.0\n",
      "20.0\n",
      "46.0\n",
      "26.0\n",
      "59.0\n",
      "30.0\n",
      "71.0\n",
      "23.0\n",
      "34.0\n",
      "34.0\n",
      "28.0\n",
      "30.0\n",
      "21.0\n",
      "33.0\n",
      "37.0\n",
      "28.0\n",
      "21.0\n",
      "30.0\n",
      "38.0\n",
      "30.0\n",
      "47.0\n",
      "30.0\n",
      "22.0\n",
      "20.0\n",
      "17.0\n",
      "21.0\n",
      "30.0\n",
      "29.0\n",
      "24.0\n",
      "2.0\n",
      "21.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "54.0\n",
      "12.0\n",
      "30.0\n",
      "24.0\n",
      "30.0\n",
      "45.0\n",
      "33.0\n",
      "20.0\n",
      "47.0\n",
      "29.0\n",
      "25.0\n",
      "23.0\n",
      "19.0\n",
      "37.0\n",
      "16.0\n",
      "24.0\n",
      "30.0\n",
      "22.0\n",
      "24.0\n",
      "19.0\n",
      "18.0\n",
      "19.0\n",
      "27.0\n",
      "9.0\n",
      "30.0\n",
      "42.0\n",
      "51.0\n",
      "22.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "51.0\n",
      "16.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "44.0\n",
      "40.0\n",
      "26.0\n",
      "17.0\n",
      "1.0\n",
      "9.0\n",
      "30.0\n",
      "45.0\n",
      "30.0\n",
      "28.0\n",
      "61.0\n",
      "4.0\n",
      "1.0\n",
      "21.0\n",
      "56.0\n",
      "18.0\n",
      "30.0\n",
      "50.0\n",
      "30.0\n",
      "36.0\n",
      "30.0\n",
      "30.0\n",
      "9.0\n",
      "1.0\n",
      "4.0\n",
      "30.0\n",
      "30.0\n",
      "45.0\n",
      "40.0\n",
      "36.0\n",
      "32.0\n",
      "19.0\n",
      "19.0\n",
      "3.0\n",
      "44.0\n",
      "58.0\n",
      "30.0\n",
      "42.0\n",
      "30.0\n",
      "24.0\n",
      "28.0\n",
      "30.0\n",
      "34.0\n",
      "30.0\n",
      "18.0\n",
      "2.0\n",
      "32.0\n",
      "26.0\n",
      "16.0\n",
      "40.0\n",
      "24.0\n",
      "35.0\n",
      "22.0\n",
      "30.0\n",
      "30.0\n",
      "31.0\n",
      "27.0\n",
      "42.0\n",
      "32.0\n",
      "30.0\n",
      "16.0\n",
      "27.0\n",
      "51.0\n",
      "30.0\n",
      "38.0\n",
      "22.0\n",
      "19.0\n",
      "30.0\n",
      "18.0\n",
      "30.0\n",
      "35.0\n",
      "29.0\n",
      "59.0\n",
      "5.0\n",
      "24.0\n",
      "30.0\n",
      "44.0\n",
      "8.0\n",
      "19.0\n",
      "33.0\n",
      "30.0\n",
      "30.0\n",
      "29.0\n",
      "22.0\n",
      "30.0\n",
      "44.0\n",
      "25.0\n",
      "24.0\n",
      "37.0\n",
      "54.0\n",
      "30.0\n",
      "29.0\n",
      "62.0\n",
      "30.0\n",
      "41.0\n",
      "29.0\n",
      "30.0\n",
      "30.0\n",
      "35.0\n",
      "50.0\n",
      "30.0\n",
      "3.0\n",
      "52.0\n",
      "40.0\n",
      "30.0\n",
      "36.0\n",
      "16.0\n",
      "25.0\n",
      "58.0\n",
      "35.0\n",
      "30.0\n",
      "25.0\n",
      "41.0\n",
      "37.0\n",
      "30.0\n",
      "63.0\n",
      "45.0\n",
      "30.0\n",
      "7.0\n",
      "35.0\n",
      "65.0\n",
      "28.0\n",
      "16.0\n",
      "19.0\n",
      "30.0\n",
      "33.0\n",
      "30.0\n",
      "22.0\n",
      "42.0\n",
      "22.0\n",
      "26.0\n",
      "19.0\n",
      "36.0\n",
      "24.0\n",
      "24.0\n",
      "30.0\n",
      "30.0\n",
      "2.0\n",
      "30.0\n",
      "50.0\n",
      "30.0\n",
      "30.0\n",
      "19.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "17.0\n",
      "30.0\n",
      "30.0\n",
      "24.0\n",
      "18.0\n",
      "26.0\n",
      "28.0\n",
      "43.0\n",
      "26.0\n",
      "24.0\n",
      "54.0\n",
      "31.0\n",
      "40.0\n",
      "22.0\n",
      "27.0\n",
      "30.0\n",
      "22.0\n",
      "30.0\n",
      "36.0\n",
      "61.0\n",
      "36.0\n",
      "31.0\n",
      "16.0\n",
      "30.0\n",
      "30.0\n",
      "38.0\n",
      "16.0\n",
      "30.0\n",
      "30.0\n",
      "29.0\n",
      "41.0\n",
      "45.0\n",
      "45.0\n",
      "2.0\n",
      "24.0\n",
      "28.0\n",
      "25.0\n",
      "36.0\n",
      "24.0\n",
      "40.0\n",
      "30.0\n",
      "3.0\n",
      "42.0\n",
      "23.0\n",
      "30.0\n",
      "15.0\n",
      "25.0\n",
      "30.0\n",
      "28.0\n",
      "22.0\n",
      "38.0\n",
      "30.0\n",
      "30.0\n",
      "40.0\n",
      "29.0\n",
      "45.0\n",
      "35.0\n",
      "30.0\n",
      "30.0\n",
      "60.0\n",
      "30.0\n",
      "30.0\n",
      "24.0\n",
      "25.0\n",
      "18.0\n",
      "19.0\n",
      "22.0\n",
      "3.0\n",
      "30.0\n",
      "22.0\n",
      "27.0\n",
      "20.0\n",
      "19.0\n",
      "42.0\n",
      "1.0\n",
      "32.0\n",
      "35.0\n",
      "30.0\n",
      "18.0\n",
      "1.0\n",
      "36.0\n",
      "30.0\n",
      "17.0\n",
      "36.0\n",
      "21.0\n",
      "28.0\n",
      "23.0\n",
      "24.0\n",
      "22.0\n",
      "31.0\n",
      "46.0\n",
      "23.0\n",
      "28.0\n",
      "39.0\n",
      "26.0\n",
      "21.0\n",
      "28.0\n",
      "20.0\n",
      "34.0\n",
      "51.0\n",
      "3.0\n",
      "21.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "33.0\n",
      "30.0\n",
      "44.0\n",
      "30.0\n",
      "34.0\n",
      "18.0\n",
      "30.0\n",
      "10.0\n",
      "30.0\n",
      "21.0\n",
      "29.0\n",
      "28.0\n",
      "18.0\n",
      "30.0\n",
      "28.0\n",
      "19.0\n",
      "30.0\n",
      "32.0\n",
      "28.0\n",
      "30.0\n",
      "42.0\n",
      "17.0\n",
      "50.0\n",
      "14.0\n",
      "21.0\n",
      "24.0\n",
      "64.0\n",
      "31.0\n",
      "45.0\n",
      "20.0\n",
      "25.0\n",
      "28.0\n",
      "30.0\n",
      "4.0\n",
      "13.0\n",
      "34.0\n",
      "5.0\n",
      "52.0\n",
      "36.0\n",
      "30.0\n",
      "30.0\n",
      "49.0\n",
      "30.0\n",
      "29.0\n",
      "65.0\n",
      "30.0\n",
      "50.0\n",
      "30.0\n",
      "48.0\n",
      "34.0\n",
      "47.0\n",
      "48.0\n",
      "30.0\n",
      "38.0\n",
      "30.0\n",
      "56.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "38.0\n",
      "33.0\n",
      "23.0\n",
      "22.0\n",
      "30.0\n",
      "34.0\n",
      "29.0\n",
      "22.0\n",
      "2.0\n",
      "9.0\n",
      "30.0\n",
      "50.0\n",
      "63.0\n",
      "25.0\n",
      "30.0\n",
      "35.0\n",
      "58.0\n",
      "30.0\n",
      "9.0\n",
      "30.0\n",
      "21.0\n",
      "55.0\n",
      "71.0\n",
      "21.0\n",
      "30.0\n",
      "54.0\n",
      "30.0\n",
      "25.0\n",
      "24.0\n",
      "17.0\n",
      "21.0\n",
      "30.0\n",
      "37.0\n",
      "16.0\n",
      "18.0\n",
      "33.0\n",
      "30.0\n",
      "28.0\n",
      "26.0\n",
      "29.0\n",
      "30.0\n",
      "36.0\n",
      "54.0\n",
      "24.0\n",
      "47.0\n",
      "34.0\n",
      "30.0\n",
      "36.0\n",
      "32.0\n",
      "38.0\n",
      "22.0\n",
      "30.0\n",
      "44.0\n",
      "30.0\n",
      "30.0\n",
      "50.0\n",
      "30.0\n",
      "39.0\n",
      "23.0\n",
      "2.0\n",
      "30.0\n",
      "17.0\n",
      "30.0\n",
      "30.0\n",
      "7.0\n",
      "45.0\n",
      "30.0\n",
      "30.0\n",
      "22.0\n",
      "36.0\n",
      "9.0\n",
      "11.0\n",
      "32.0\n",
      "50.0\n",
      "64.0\n",
      "19.0\n",
      "30.0\n",
      "33.0\n",
      "8.0\n",
      "17.0\n",
      "27.0\n",
      "30.0\n",
      "22.0\n",
      "22.0\n",
      "62.0\n",
      "48.0\n",
      "30.0\n",
      "39.0\n",
      "36.0\n",
      "30.0\n",
      "40.0\n",
      "28.0\n",
      "30.0\n",
      "30.0\n",
      "24.0\n",
      "19.0\n",
      "29.0\n",
      "30.0\n",
      "32.0\n",
      "62.0\n",
      "53.0\n",
      "36.0\n",
      "30.0\n",
      "16.0\n",
      "19.0\n",
      "34.0\n",
      "39.0\n",
      "30.0\n",
      "32.0\n",
      "25.0\n",
      "39.0\n",
      "54.0\n",
      "36.0\n",
      "30.0\n",
      "18.0\n",
      "47.0\n",
      "60.0\n",
      "22.0\n",
      "30.0\n",
      "35.0\n",
      "52.0\n",
      "47.0\n",
      "30.0\n",
      "37.0\n",
      "36.0\n",
      "30.0\n",
      "49.0\n",
      "30.0\n",
      "49.0\n",
      "24.0\n",
      "30.0\n",
      "30.0\n",
      "44.0\n",
      "35.0\n",
      "36.0\n",
      "30.0\n",
      "27.0\n",
      "22.0\n",
      "40.0\n",
      "39.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "35.0\n",
      "24.0\n",
      "34.0\n",
      "26.0\n",
      "4.0\n",
      "26.0\n",
      "27.0\n",
      "42.0\n",
      "20.0\n",
      "21.0\n",
      "21.0\n",
      "61.0\n",
      "57.0\n",
      "21.0\n",
      "26.0\n",
      "30.0\n",
      "80.0\n",
      "51.0\n",
      "32.0\n",
      "30.0\n",
      "9.0\n",
      "28.0\n",
      "32.0\n",
      "31.0\n",
      "41.0\n",
      "30.0\n",
      "20.0\n",
      "24.0\n",
      "2.0\n",
      "30.0\n",
      "30.0\n",
      "48.0\n",
      "19.0\n",
      "56.0\n",
      "30.0\n",
      "23.0\n",
      "30.0\n",
      "18.0\n",
      "21.0\n",
      "30.0\n",
      "18.0\n",
      "24.0\n",
      "30.0\n",
      "32.0\n",
      "23.0\n",
      "58.0\n",
      "50.0\n",
      "40.0\n",
      "47.0\n",
      "36.0\n",
      "20.0\n",
      "32.0\n",
      "30.0\n",
      "30.0\n",
      "43.0\n",
      "30.0\n",
      "40.0\n",
      "31.0\n",
      "70.0\n",
      "31.0\n",
      "30.0\n",
      "18.0\n",
      "30.0\n",
      "18.0\n",
      "38.0\n",
      "36.0\n",
      "30.0\n",
      "27.0\n",
      "20.0\n",
      "14.0\n",
      "60.0\n",
      "25.0\n",
      "14.0\n",
      "19.0\n",
      "18.0\n",
      "15.0\n",
      "31.0\n",
      "4.0\n",
      "30.0\n",
      "25.0\n",
      "60.0\n",
      "52.0\n",
      "44.0\n",
      "30.0\n",
      "49.0\n",
      "42.0\n",
      "18.0\n",
      "35.0\n",
      "18.0\n",
      "25.0\n",
      "26.0\n",
      "39.0\n",
      "45.0\n",
      "42.0\n",
      "22.0\n",
      "30.0\n",
      "24.0\n",
      "30.0\n",
      "48.0\n",
      "29.0\n",
      "52.0\n",
      "19.0\n",
      "38.0\n",
      "27.0\n",
      "30.0\n",
      "33.0\n",
      "6.0\n",
      "17.0\n",
      "34.0\n",
      "50.0\n",
      "27.0\n",
      "20.0\n",
      "30.0\n",
      "30.0\n",
      "25.0\n",
      "25.0\n",
      "29.0\n",
      "11.0\n",
      "30.0\n",
      "23.0\n",
      "23.0\n",
      "30.0\n",
      "48.0\n",
      "35.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "36.0\n",
      "21.0\n",
      "24.0\n",
      "31.0\n",
      "70.0\n",
      "16.0\n",
      "30.0\n",
      "19.0\n",
      "31.0\n",
      "4.0\n",
      "6.0\n",
      "33.0\n",
      "23.0\n",
      "48.0\n",
      "30.0\n",
      "28.0\n",
      "18.0\n",
      "34.0\n",
      "33.0\n",
      "30.0\n",
      "41.0\n",
      "20.0\n",
      "36.0\n",
      "16.0\n",
      "51.0\n",
      "30.0\n",
      "30.0\n",
      "30.0\n",
      "32.0\n",
      "24.0\n",
      "48.0\n",
      "57.0\n",
      "30.0\n",
      "54.0\n",
      "18.0\n",
      "30.0\n",
      "5.0\n",
      "30.0\n",
      "43.0\n",
      "13.0\n",
      "17.0\n",
      "29.0\n",
      "30.0\n",
      "25.0\n",
      "25.0\n",
      "18.0\n",
      "8.0\n",
      "1.0\n",
      "46.0\n",
      "30.0\n",
      "16.0\n",
      "30.0\n",
      "30.0\n",
      "25.0\n",
      "39.0\n",
      "49.0\n",
      "31.0\n",
      "30.0\n",
      "30.0\n",
      "34.0\n",
      "31.0\n",
      "11.0\n",
      "30.0\n",
      "27.0\n",
      "31.0\n",
      "39.0\n",
      "18.0\n",
      "39.0\n",
      "33.0\n",
      "26.0\n",
      "39.0\n",
      "35.0\n",
      "6.0\n",
      "30.0\n",
      "30.0\n",
      "23.0\n",
      "31.0\n",
      "43.0\n",
      "10.0\n",
      "52.0\n",
      "27.0\n",
      "38.0\n",
      "27.0\n",
      "2.0\n",
      "30.0\n",
      "30.0\n",
      "1.0\n",
      "30.0\n",
      "62.0\n",
      "15.0\n",
      "30.0\n",
      "30.0\n",
      "23.0\n",
      "18.0\n",
      "39.0\n",
      "21.0\n",
      "30.0\n",
      "32.0\n",
      "30.0\n",
      "20.0\n",
      "16.0\n",
      "30.0\n",
      "30.0\n",
      "17.0\n",
      "42.0\n",
      "30.0\n",
      "35.0\n",
      "28.0\n",
      "30.0\n",
      "4.0\n",
      "74.0\n",
      "9.0\n",
      "16.0\n",
      "44.0\n",
      "18.0\n",
      "45.0\n",
      "51.0\n",
      "24.0\n",
      "30.0\n",
      "41.0\n",
      "21.0\n",
      "48.0\n",
      "30.0\n",
      "24.0\n",
      "42.0\n",
      "27.0\n",
      "31.0\n",
      "30.0\n",
      "4.0\n",
      "26.0\n",
      "47.0\n",
      "33.0\n",
      "47.0\n",
      "28.0\n",
      "15.0\n",
      "20.0\n",
      "19.0\n",
      "30.0\n",
      "56.0\n",
      "25.0\n",
      "33.0\n",
      "22.0\n",
      "28.0\n",
      "25.0\n",
      "39.0\n",
      "27.0\n",
      "19.0\n",
      "30.0\n",
      "26.0\n",
      "32.0\n",
      "38.0\n"
     ]
    }
   ],
   "source": [
    "titanic_clear_columns['Age'] = titanic_clear_columns['Age'].fillna(round(titanic_clear_columns['Age'].mean()))\n",
    "for index, row in titanic_clear_columns.iterrows():\n",
    "    print(row[\"Age\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejnym etapem będzie usunięcie kolumny **Cabin**.<br>\n",
    "W wyniku analizy ustalono, że wartości kolumny to w większości wartości zerowe, poza tym nie przynoszą one żadnej pożytecznej wiedzy, dodatkowo będą negatywnie wpływały na optymalizację uczenia maszynowego."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7,25</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71,2833</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7,925</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53,1</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8,05</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Survived  Pclass  \\\n",
       "PassengerId                     \n",
       "1                   0       3   \n",
       "2                   1       1   \n",
       "3                   1       3   \n",
       "4                   1       1   \n",
       "5                   0       3   \n",
       "\n",
       "                                                          Name     Sex   Age  \\\n",
       "PassengerId                                                                    \n",
       "1                                      Braund, Mr. Owen Harris    male  22.0   \n",
       "2            Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "3                                       Heikkinen, Miss. Laina  female  26.0   \n",
       "4                 Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "5                                     Allen, Mr. William Henry    male  35.0   \n",
       "\n",
       "             SibSp  Parch            Ticket     Fare Embarked  \n",
       "PassengerId                                                    \n",
       "1                1      0         A/5 21171     7,25        S  \n",
       "2                1      0          PC 17599  71,2833        C  \n",
       "3                0      0  STON/O2. 3101282    7,925        S  \n",
       "4                1      0            113803     53,1        S  \n",
       "5                0      0            373450     8,05        S  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_clear_columns.drop('Cabin',axis=1, inplace=True)\n",
    "titanic_clear_columns.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejnym etapem będzie uzupełnienie brakujących wartości kolumny **Embarked**.<br>\n",
    "Jako że brakujących wartości jest tylko 2, wybrano losowo wartość 'S' i wpisano do brakujących pól."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_clear_columns['Embarked'] = titanic_clear_columns['Embarked'].fillna('S')\n",
    "titanic_clear_columns.isnull().values.any()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wynik 'False\" ostatniego bloku oznacza, że kolumna 'Embarked' nie zawiera już pustych wartości."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejnym etapem będzie sprawdzenie unikalnych wartości kolumny **Sex**.<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['male' 'female' 'malef' 'mal' 'fem' 'femmale']\n"
     ]
    }
   ],
   "source": [
    "unkalna_plec = titanic_clear_columns['Sex'].unique()\n",
    "print(unkalna_plec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Powyższy wynik podsuwa wniosek, że w danych wystąpiły literówki. Zamieniono więc błędne wartości na poprawne, najbliżej brzmiące."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['male' 'female']\n"
     ]
    }
   ],
   "source": [
    "titanic_clear_columns['Sex'] = titanic_clear_columns['Sex'].replace(['malef'],'male')\n",
    "titanic_clear_columns['Sex'] = titanic_clear_columns['Sex'].replace(['mal'],'male')\n",
    "titanic_clear_columns['Sex'] = titanic_clear_columns['Sex'].replace(['fem'],'female')\n",
    "titanic_clear_columns['Sex'] = titanic_clear_columns['Sex'].replace(['femmale'],'female')\n",
    "unkalna_plec = titanic_clear_columns['Sex'].unique()\n",
    "print(unkalna_plec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejnym etapem będzie uzupełnienie brakujących lub błędnych danych w kolumnie **Fare**, na sposób podobny jak w przypadku kolumny Age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Survived      int64\n",
       "Pclass        int64\n",
       "Name         object\n",
       "Sex          object\n",
       "Age         float64\n",
       "SibSp         int64\n",
       "Parch         int64\n",
       "Ticket       object\n",
       "Fare        float64\n",
       "Embarked     object\n",
       "dtype: object"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for i, row in titanic_clear_columns.iterrows():\n",
    "    try:\n",
    "        titanic_clear_columns.at[i, 'Fare'] = float(row['Fare'])       \n",
    "    except ValueError:\n",
    "        titanic_clear_columns.at[i, 'Fare'] = np.NaN\n",
    "\n",
    "titanic_clear_columns[['Fare']] = titanic_clear_columns[['Fare']].astype(float).round(2)\n",
    "titanic_clear_columns.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "W powyższym wyniku można zauważyć, że kolumna **Fare** zmieniła typ danych na liczbowy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kolejnym etapem będzie sprawdzenie, czy występują zduplikowane rekordy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId\n",
       "1       False\n",
       "2       False\n",
       "3       False\n",
       "4       False\n",
       "5       False\n",
       "        ...  \n",
       "888     False\n",
       "889     False\n",
       "890     False\n",
       "891     False\n",
       "1000    False\n",
       "Length: 892, dtype: bool"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_clear_columns.duplicated(keep='first')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Jak widać duplikaty występują (np. dla pasażera o id 11). Kolejnym krokiem będzie ich usunięcie."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PassengerId\n",
       "1       False\n",
       "2       False\n",
       "3       False\n",
       "4       False\n",
       "5       False\n",
       "        ...  \n",
       "888     False\n",
       "889     False\n",
       "890     False\n",
       "891     False\n",
       "1000    False\n",
       "Length: 887, dtype: bool"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_clear_columns.drop_duplicates(keep = False, inplace = True)\n",
    "titanic_clear_columns.duplicated(keep='first')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wykonanie powyższych kroków pozwoli na oczyszczenie i przygotowanie do analizy większości zbiorów danych.<br>\n",
    "Przygotowane w ten sposób dane pozwolą na dalszą analizę, budowę modeli dla zbioru oraz wykorzystanie uczenia maszynowego do przewidywania danych."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ostatnim etapem sprawozdania jest eksport wyczyszczonego zbioru do pliku **TitanicCleaned.tsv**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic_clear_columns.to_csv('TitanicCleaned.tsv', sep = '\\t')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}