Canadian Climate Data Scraping Tool
Charmaine Bonifacio (a), Thomas E. Barchyn (b), Chris H. Hugenholtz (b), Stefan W. Kienzle (a,c)
(a) Department of Geography, University of Lethbridge, 4401 University Drive, Lethbridge, AB, Canada T1K 3M4 (b) Department of Geography, University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4 (c) Applied Behavioural Ecology and Ecosystems Research Unit, University of South Africa, PO Box 392, Florida, Pretoria, South Africa
In this paper we present a new software tool that automatically fetches, downloads and consolidates climate data from a Web database where the data are contained on multiple Web pages. The tool is called the Canadian Climate Data Scraping Tool (CCDST) and was developed to enhance access and simplify analysis of climate data from Canada's National Climate Data and Information Archive (NCDIA). The CCDST deconstructs a URL for a particular climate station in the NCDIA and then iteratively modifies the date parameters to download large volumes of data, remove individual file headers, and merge data files into one output file. This automated sequence enhances access to climate data by substantially reducing the time needed to manually download data from multiple Web pages. To this end, we present a case study of the temporal dynamics of blowing snow events that resulted in ~3.1 weeks time savings. Without the CCDST, the time involved in manually downloading climate data limits access and restrains researchers and students from exploring climate trends. The tool is coded as a Microsoft Excel macro and is available to researchers and students for free. The main concept and structure of the tool can be modified for other Web databases hosting geophysical data.
- Climate data online
- Scraping tool
- Canada
Please download the user documentation and macro from the Download folder. These supporting documents were last update May 25, 2016.
The Canadian Climate Scraping Tool is licensed under a GNU General Public License v2.0
Copyright (C) 2013 Charmaine Michelle T. Bonifacio
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.