Lab11-Exercises.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# k-Means\n",
    "\n",
    "## Exercise 1\n",
    "k-Means on two-dimensional data with cluster separators\n",
    "\n",
    "For the dataset below, plot the clusters and the centroids of the k-means algorithm for each iteration until convergence. The initial centroids will be the points A, D and G (therefore the algorithm will find 3 clusters). Include in each plot the Voronoi diagram for the centroids, to highlight the cluster separation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "data = {\n",
    "    'A':[2, 10], 'B':[2, 5], 'C':[8, 4], 'D':[5, 8], \n",
    "    'E':[7, 5], 'F':[6, 4],  'G':[1, 2], 'H':[4, 9]\n",
    "}\n",
    "d = pd.DataFrame.from_dict(data, orient='index', columns=['X', 'Y'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2\n",
    "k-Means on two-dimensional data\n",
    "\n",
    "For the dataset below and the initial centroids A, D and G, independently implement the k-Means algorithm (i.e. do not use the one from `sklearn`) and plot the clusters and centroids for each iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "data = {\n",
    "    'A':[2, 10], 'B':[2, 5], 'C':[8, 4], 'D':[5, 8], \n",
    "    'E':[7, 5], 'F':[6, 4],  'G':[1, 2], 'H':[4, 9]\n",
    "}\n",
    "d = pd.DataFrame.from_dict(data, orient='index', columns=['X', 'Y'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3\n",
    "k-Means on an external dataset with starting centroids\n",
    "\n",
    "Apply k-means on this [two-dimensional dataset](https://profs.info.uaic.ro/~ciortuz/ML.ex-book/res/CMU.2004f.TM+AM.HW3.pr5.cl.dat) using these [starting centroids](https://profs.info.uaic.ro/~ciortuz/ML.ex-book/res/CMU.2004f.TM+AM.HW3.pr5.init.dat). Plot the clusters and centroids after each iteration until convergence. What is unusual about about the first iteration?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4\n",
    "Anisotropically distributed data\n",
    "\n",
    "Run the k-means algorithm for the datasets `d1` and `d2` with $k=3$ and the default parameters. \n",
    "1. Plot the resulting clusters.\n",
    "1. Which clusters look more 'natural' and why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "import numpy as np\n",
    "\n",
    "n_samples = 1500\n",
    "random_state = 170\n",
    "X, y = make_blobs(n_samples=n_samples, random_state=random_state)\n",
    "# Anisotropically distributed data\n",
    "transformation = [[0.60834549, -0.63667341], [-0.40887718, 0.85253229]]\n",
    "anis = np.dot(X, transformation)\n",
    "# Compare these datasets\n",
    "d1, d2 = X, anis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5\n",
    "k-means and noise\n",
    "\n",
    "Considering the dataset `d` below and two sets of starting centroids `c1` and `c2`.\n",
    "\n",
    "1. Run k-means ($k=3$ and the default parameters), first starting with `c1` and then starting with `c2`. (You might want to also use `n_init=1` to prevent a warning.)\n",
    "1. Plot the resulting clusters for each of the two runs.\n",
    "1. In which of the two runs the clusters look more 'natural' and why?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "import numpy as np\n",
    "\n",
    "n_samples = 1500\n",
    "random_state = 110\n",
    "d, _ = make_blobs(n_samples=n_samples, random_state=random_state)\n",
    "# Dataset\n",
    "d = np.append(d, [[-10, 15]], axis=0)\n",
    "# Starting centroids\n",
    "c1 = np.array([[-6, 2], [-10, 15], [3, 3]])\n",
    "c2 = np.array([[-10, 3], [-2, 2], [3, 3]])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}