Lab12-Exercises.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# k-Means\n",
    "\n",
    "## Exercise 1\n",
    "Intra-cluster cohesion increase\n",
    "\n",
    "For the dataset `d` and `start_centroids` below, show that the $J$ criterion (or inertia) monotonically decreases for each successive iteration of the algorithm by plotting it on a line chart using `pyplot.plot`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "n_samples = 1500\n",
    "random_state = 170\n",
    "X, y = make_blobs(n_samples=n_samples, random_state=random_state)\n",
    "d = pd.DataFrame(X, columns=['X1', 'X2'])\n",
    "start_centroids = np.array([[0, 0.1], [0, 0.2], [0, 0.3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2\n",
    "Intra-cluster cohension calculation\n",
    "\n",
    "Given the dataset `d` and `start_centroids` below:\n",
    "1. Run the k-means algorithm form `sklearn` on the dataset `d` and plot the resulting clusters using a scatterplot.\n",
    "1. Print the intra-cluster cohesion as computed by the algorithm for the resulting clusters.\n",
    "1. Independently calculate the intra-cluster cohesion for the resulting clusters (it should match the one computed by the algorithm)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "n_samples = 1500\n",
    "random_state = 110\n",
    "X, y = make_blobs(n_samples=n_samples, random_state=random_state)\n",
    "d = pd.DataFrame(X, columns=['X1', 'X2'])\n",
    "start_centroids = np.array([[0, 0.1], [0, 0.2], [0, 0.3]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3\n",
    "k-means++\n",
    "\n",
    "Show that _k-means++_ provides better centroid initialisation by comparing the average value of $J$ (i.e. the `inertia_` attribute) for _random_ initializations with _k-means++_ initialisations. More specifically:\n",
    "* on the dataset `d` below, run the k-means algorithm 1000 times using _random_ initialisation and record the `inertia_` attribute. Plot the histogram of the recorded values and print their mean;\n",
    "* repeat the process using _k-means++_;\n",
    "* Which method performs better?\n",
    "\n",
    "For the `KMeans` function, make sure to always use the parameters `max_iter=1, n_init=1, algorithm='full', n_clusters=3, random_state=None` to emphasize the effect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "n_samples = 1500\n",
    "random_state = 100\n",
    "X, y = make_blobs(n_samples=n_samples, random_state=random_state)\n",
    "d = pd.DataFrame(X, columns=['X1', 'X2'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4\n",
    "k-means++ calculation\n",
    "\n",
    "Given a dataset containing the points A(-1, 0), B(1, 0), C(0, 1), D(3, 0) and E(3, 1):\n",
    "1. Plot the points using a scatterplot.\n",
    "1. Considering the k-means algorithm with $k=2$ and `random` initialisation. During the initial centroid selection (before the first iteration), if the first centroid was chosen at random to be the point A, what is the probability that the next centroid will be chosen from the set {B, C}?\n",
    "1. Calculate the same probability, but this time for `k-means++` instead of `random`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5\n",
    "The best value for $k$\n",
    "\n",
    "For the dataset `d` below, find the number of clusters for the k-means algorithm using the \"elbow\" method. Make sure to plot the points using a scatterplot and the line chart for the $J$ criterion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "n_samples = 1500\n",
    "random_state = 160\n",
    "X, y = make_blobs(n_samples=n_samples, centers=6, random_state=random_state)\n",
    "d = pd.DataFrame(X, columns=['X1', 'X2'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}