eurovision_demo.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import base_features\n",
    "import pitch_features\n",
    "import feature_transforms\n",
    "import utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Catchy feature extraction\n",
    "\n",
    "## Outline\n",
    "\n",
    "This notebook shows how to compute features for a set of presegmented audiofiles.\n",
    "\n",
    "Extracting catchy features from a folder of such files involves three steps:\n",
    "\n",
    "#### 1. Base feature extraction\n",
    "\n",
    "Here, basic, familiar feature time series are extracted. The toolbox currently implements (wrappers for) MFCC, chroma, melody and perceptual feature extraction.\n",
    "\n",
    "This part of the toolbox relies on a lot of external code, but it's also easy to work around: if you want to use other features, just save them to a set of csv files (1 per song section--see below) in some folder (1 per feature).\n",
    "\n",
    "#### 2. Pitch descriptor extraction\n",
    "This part computes mid-level pitch descriptors from chroma and/or melody information computed in step one.\n",
    "Essentially an implementation of several kinds of audio bigram descriptors.\n",
    "\n",
    "See also [1] and [2].\n",
    "\n",
    "#### 3. Feature transforms\n",
    "Compute 'first' and 'second order' aggregates of any of the features computed in step 1 and step 2.\n",
    "\n",
    "See [2].\n",
    "\n",
    "[1] Van Balen, J., Wiering, F., & Veltkamp, R. (2015). Audio Bigrams as a Unifying Model of Pitch-based Song Description. In Proc. 11th International Symposium on Computer Music Multidisciplinary Research (CMMR). Plymouth, United Kingdom.\n",
    "\n",
    "[2] Van Balen, J., Burgoyne, J. A., Bountouridis, D., Müllensiefen, D., & Veltkamp, R. (2015). Corpus Analysis Tools for Computational Hook Discovery. In Proc. 16th International Society for Music Information Retrieval Conference (pp. 227–233). Malaga, Spain.\n",
    "\n",
    "## Dataset\n",
    "\n",
    "Let's import some audio data and see how all of this works.\n",
    "\n",
    "The CATCHY toolbox was designed for the analysis of a corpus of song *sections*.\n",
    "\n",
    "CATCHY therefore requires data to be represented as a python dictionary of song section paths, grouped by song id.\n",
    "\n",
    "`utils.dataset_from_dir()` makes such a dictionary given a folder of audio files, labeled `songid-sectionid.ext` where `ext` can be `wav` or `mp3`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'utils' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-0741743ba8a7>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0maudio_dir\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'../Cogitch/Audio/Eurovision/'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mutils\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdataset_from_dir\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maudio_dir\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'utils' is not defined"
     ]
    }
   ],
   "source": [
    "audio_dir = '../Cogitch/Audio/Eurovision/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "euro_dict = utils.dataset_from_dir(audio_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Base features\n",
    "\n",
    "Basic feature time series can be extracted using the `base_features` module.\n",
    "The function `compute_and_write()` provides a convenient wrapper around most of the functionality in this module, reading audio and computing a set of basic, useful features.\n",
    "\n",
    "The results will be written to a set of csv files in `data_dir`.\n",
    "\n",
    "Currently requires a dir to made for each of the features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data_dir = '../Cogitch/Data/Eurovision/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# base_features.compute_and_write(audio_dir, data_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pitch Features\n",
    "\n",
    "\n",
    "The `pitch_features` module provides code to compute, from the variable-length base features computed above, fixed-sized melody and harmony descriptors for each of the song sections.\n",
    "\n",
    "`pitch_features.compute_and_write()` again provides a high-level wrapper function.\n",
    "The features that it should compute must be provided in a dictionary of `(feature_function, parameters)` tuples, with some feature name of your choice for each as keys.\n",
    "\n",
    "The result is again stored in a set of csv files. Directories are the feature names provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pitch_features.melody_dir = data_dir + 'melody/'\n",
    "pitch_features.chroma_dir = data_dir + 'hpcp/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "features = {'pitchhist3': (pitch_features.get_pitchhist3, {}),\n",
    "            'pitchhist3_int': (pitch_features.get_pitchhist3, {'intervals': True}),\n",
    "            'chromahist3': (pitch_features.get_chromahist3, {}),\n",
    "            'chromahist3_int': (pitch_features.get_chromahist3, {'intervals': True}),\n",
    "            'harmonisation': (pitch_features.get_harmonisation, {}),\n",
    "            'harmonisation_int': (pitch_features.get_harmonisation, {'intervals': True}) }\n",
    "\n",
    "# pitch_features.compute_and_write(data_dir, features=features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Transforms\n",
    "\n",
    "The `feature_transforms` module allows you to compute first- and second-order features based on any of the features above. The transforms to be applied must be passed to the `compute()` function using a special syntax. The syntax states a feature, a reference corpus, and an aggregation function.\n",
    "\n",
    "From the doc string:\n",
    "\n",
    "    - feature name and aggregates are separated by dots, e.g. 'mfcc.entropy'\n",
    "    - feature name is first and contains no dots\n",
    "    - first order and second order aggregates are separated by one of 2 keywords:\n",
    "        'corpus' or 'song'\n",
    "\n",
    "    Ex.:\n",
    "    >>> parse_features('loudness.mean.song.pdf.log')\n",
    "    ('loudness', ['mean'], ['song', 'pdf', 'log'])\n",
    "    \n",
    "The above shows how the transform names are read. In the example:\n",
    "\n",
    "    `loudness.mean.song.pdf.log` \n",
    "\n",
    "computes the log of the probability density function of the distribution of the loudness features' mean within the song (i.e., across the sections of the song).\n",
    "\n",
    "\n",
    "The result is returned in a Pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "feature_transforms.data_dir = data_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "source": [
    "The above tells the module where to look for base features.\n",
    "\n",
    "Below, a set of tested first and second-order features is computed for the full dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "features = [\n",
    "'harmonisation_int.corpus.information',\n",
    "'harmonisation_int.corpus.tau',\n",
    "'harmonisation_int.song.information',\n",
    "'harmonisation_int.song.tau',\n",
    "'harmonisation.normentropy.minlog',\n",
    "'harmonisation.normentropy.minlog.corpus.pdf.rank.logit',\n",
    "'harmonisation.normentropy.minlog.song.pdf.rank.logit',\n",
    "'chromahist3_int.corpus.information',\n",
    "'chromahist3_int.corpus.tau',\n",
    "'chromahist3_int.song.information',\n",
    "'chromahist3_int.song.tau',\n",
    "'chromahist3.normentropy.minlog',\n",
    "'chromahist3.normentropy.minlog.corpus.pdf.rank.logit',\n",
    "'chromahist3.normentropy.minlog.song.pdf.rank.logit',\n",
    "'loudness.mean',\n",
    "'loudness.mean.corpus.pdf.rank.logit',\n",
    "'loudness.mean.song.pdf.rank.logit',\n",
    "'loudness.std',\n",
    "'loudness.std.corpus.pdf.rank.logit',\n",
    "'loudness.std.song.pdf.rank.logit',\n",
    "'pitchhist3_int.corpus.information',\n",
    "'pitchhist3_int.corpus.tau',\n",
    "'pitchhist3_int.song.information',\n",
    "'pitchhist3_int.song.tau',\n",
    "'pitchhist3.normentropy.minlog',\n",
    "'pitchhist3.normentropy.minlog.corpus.pdf.rank.logit',\n",
    "'pitchhist3.normentropy.minlog.song.pdf.rank.logit',\n",
    "'mfcc.mean.corpus.indeppdf.rank.logit',\n",
    "'mfcc.mean.song.indeppdf.rank.logit',\n",
    "'mfcc.totvar.log',\n",
    "'mfcc.totvar.log.corpus.pdf.rank.logit',\n",
    "'mfcc.totvar.log.song.pdf.rank.logit',\n",
    "'melody.mean',\n",
    "'melody.mean.corpus.pdf.rank.logit',\n",
    "'melody.mean.song.pdf.rank.logit',\n",
    "'melody.std.log',\n",
    "'melody.std.log.corpus.pdf.rank.logit',\n",
    "'melody.std.log.song.pdf.rank.logit',\n",
    "'roughness.mean.log',\n",
    "'roughness.mean.log.corpus.pdf.rank.logit',\n",
    "'roughness.mean.log.song.pdf.rank.logit',\n",
    "'sharpness.mean',\n",
    "'sharpness.mean.corpus.pdf.rank.logit',\n",
    "'sharpness.mean.song.pdf.rank.logit']\n",
    "\n",
    "data = feature_transforms.compute(euro_dict, features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Output\n",
    "\n",
    "Finally, output data to a single CSV file for use in another notebook or R."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# data.hist(figsize=(28,21));\n",
    "data.to_csv('euro_features.csv', index=None)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}