04-fits.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reading and writing FITS files\n",
    "\n",
    "The [astropy.io.fits](http://docs.astropy.org/en/stable/io/fits/index.html) sub-package provides a way to read, write, and manipulate FITS files. It is one of the oldest packages in Astropy, and started out as the separate PyFITS package."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<section class=\"objectives panel panel-warning\">\n",
    "<div class=\"panel-heading\">\n",
    "<h2><span class=\"fa fa-certificate\"></span> Objectives</h2>\n",
    "</div>\n",
    "\n",
    "\n",
    "<div class=\"panel-body\">\n",
    "\n",
    "<ul>\n",
    "<li>Read image FITS files and access data and header</li>\n",
    "<li>Read tabular FITS files and access data and header</li>\n",
    "<li>Construct new FITS files from scratch</li>\n",
    "</ul>\n",
    "\n",
    "</div>\n",
    "\n",
    "</section>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Documentation\n",
    "\n",
    "This notebook only shows a subset of the functionality in astropy.io.fits. For more information about the features presented below as well as other available features, you can read the\n",
    "[astropy.io.fits documentation](https://docs.astropy.org/en/stable/io/fits/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rc('image', origin='lower')\n",
    "plt.rc('figure', figsize=(10, 6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "\n",
    "For this tutorial we will be using the ``LMCDensFits1k.fits`` FITS file which was downloaded from http://www.sim.ul.pt/gaia/dr1/gallery/. This file contains the density of sources in the Gaia DR1 release towards the Large Magellanic Cloud (LMC). In addition, we will use the ``gaia_lmc_psc.fits`` file which contains the result of a table query in the GAIA archive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading FITS files and accessing data\n",
    "\n",
    "To open a FITS file, use the ``fits.open`` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The returned object, ``hdulist``, behaves like a Python list, and each element maps to a Header-Data Unit (HDU) in the FITS file. You can view more information about the FITS file with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, this file contains only one HDU. Accessing this HDU can then either be done by index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or by name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``hdu`` object then has two important attributes: ``data``, which behaves like a Numpy array, can be used to access the data, and ``header``, which behaves like a dictionary, can be used to access the header information. First, we can take a look at the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tells us that it is a 2-d image. We can now take a peak at the header:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which shows that this is an Orthographic (``-SIN``) projection in Galactic Coordinates. We can access individual\n",
    "header keywords using standard item notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now take a look at the data using Matplotlib:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this is just a plot of an array, so the coordinates are just pixel coordinates at this stage.\n",
    "\n",
    "Modifying data or header information in a FITS file object is easy. For example we can add new keywords with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also modify the data by extracting a subset for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and updating the CRPIX values accordingly in the header:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now write out the HDU list back to disk:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or if you were dealing with a multi-HDU file and wanted to just output the HDU you were editing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a FITS file from scratch\n",
    "\n",
    "If you want to create a FITS file from scratch, you need to start off by creating an HDU object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and you can then populate the data and header attributes with whatever information you like, for example random data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that setting the data automatically populates the header with basic information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and you should never have to set header keywords such as ``NAXIS``, ``NAXIS1``, and so on manually. We can then set additional header keywords:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and we can then write out the FITS file to disk:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convenience functions\n",
    "\n",
    "In cases where you just want to access the data or header in a specific HDU, you can use the following convenience functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can optionally specify the index or name of the HDU to look at (which is important if you are dealing with a multi-HDU file):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and similarly for ``getheader``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing Tabular Data\n",
    "\n",
    "Let's now take a look at a tabular dataset - this is a subset of sources from the GAIA DR1 release around the LMC, and only includes the brightest sources in Gmag:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have two HDUs - the first one is mostly empty and the second contains the table (the primary HDU can't contain a table in general).\n",
    "\n",
    "Tabular data behaves very similarly to image data such as that shown above, but the data array is a structured Numpy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<section class=\"challenge panel panel-success\">\n",
    "<div class=\"panel-heading\">\n",
    "<h2><span class=\"fa fa-pencil\"></span> Challenge</h2>\n",
    "</div>\n",
    "\n",
    "\n",
    "<div class=\"panel-body\">\n",
    "\n",
    "<ol>\n",
    "<li>Examine the headers of the first and second HDU in the point source catalog and try adding new keywords to them or changing them</li>\n",
    "<li>Make a histogram of the G-band magnitude of the sources in the catalog - can you figure out what the upper limit for Gmag was when the table was selected?</li>\n",
    "<li>Make a plot of the position of the sources on the sky in the point source catalog, in Galactic coordinates l and b (you will need to use what we learned about SkyCoord in the previous tutorial)</li>\n",
    "<li>Try and produce a new FITS file that contains the image in the primary HDU and the above table in the second HDU</li>\n",
    "</ol>\n",
    "\n",
    "</div>\n",
    "\n",
    "</section>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><i>This notebook was written by <a href=\"https://aperiosoftware.com/\">Aperio Software Ltd.</a> &copy; 2019, and is licensed under a <a href=\"https://creativecommons.org/licenses/by/4.0/\">Creative Commons Attribution 4.0 International License (CC BY 4.0)</a></i></center>\n",
    "\n",
    "![cc](https://mirrors.creativecommons.org/presskit/buttons/88x31/svg/by.svg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}