UCL Geography: Level 7 course, Scientific Computing
To install the software for this course, first make sure you have pip and git installed on your computer.
For example, if using conda, at the command line, type:
conda install git pip
Then use pip to install from github:
pip install git+git://github.com/profLewis/newform0111@master
conda config --add channels profLewis
Run the notebooks on Binder server directly by click on different chaper, it may take some time to start but just wait a bit....
Go to individual chapter:
Chapter1_Python_introduction_answers
Chapter2_Numpy_matplotlib_answers
Chapter3_2_MODIS_download_answers
Chapter3_4_GDAL_stacking_and_interpolating
Chapter3_4a_GDAL_stacking_and_interpolating-convolution
Chapter3_6A_GDAL_Reconciling_projections_prerequisites
Chapter3_6_GDAL_Reconciling_projections
Chapter5_Modelling_and_optimisation
Chapter6_NonLinear_Model_Fitting
Chapter6_NonLinear_Model_Fitting_Solutions
Chapter7_FittingPhenologyModels
Chapter7_FittingPhenologyModels_Solutions
Chapter9_Fire_and_Teleconnections
Chapter9_Fire_and_Teleconnections_Solution
N.B. 2019-20 Course Convenors: Dr Qingling Wu
This course, newform0111 Scientific Computing, is a term 1 MSc module worth 15 credits (25% of the term 1 credits) that aims to:
- impart an understanding of scientific computing
- give students a grounding in the basic principles of algorithm development and program construction
- to introduce principles of computer-based image analysis and model development
It is open to students from a number of MSc courses run by the Department of Geography UCL, but the material should be of wider value to others wishing to make use of scientific computing.
The module will cover:
- Computing in Python
- Computing for image analysis
- Computing for environmental modelling
- Data visualisation for scientific applications
At the end of the module, students should:
- have an understanding of the Python programmibng language and experience of its use
- have an understanding of algorithm development and be able to use widely used scientific computing software to manipulate datasets and accomplish analytical tasks
- have an understanding of the technical issues specific to image-based analysis, model implementation and scientific visualisation
The course takes place over 10 weeks in term 1, in the Geography Department Unix Computing Lab (PB110) in the Pearson Building, UCL.
Classes take place from the second week of term to the final week of term, other than Reading week. See UCL term dates for further information.
The timetable is available on the UCL Academic Calendar
Assessment is through two pieces of coursework, submitted in both paper form and electronically via Moodle.
See the Moodle page for more details.
Python is a high level programming language that is freely available, relatively easy to learn and portable across different computing systems. In Python, you can rapidly develop solutions for the sorts of problems you might need to solve in your MSc courses and in the world beyond. Code written in Python is also easy to maintain, is (or should be) self-documented, and can easily be linked to code written in other languages.
Relevant features include:
- it is automatically compiled and executed
- code is portable provided you have the appropriate Python modules.
- for compute intensive tasks, you can easily make calls to methods written in (faster) lower-level languages such as C or FORTRAN
- there is an active user and development community, which means that new capabilities appear over time and there are many existing extensions and enhancements easily available to you.
For further background on Python, look over the material on Advanced Scientific Programming in Python or python.org web sites.
We are assuming that you are new to computing in this course. We will not explicitly go through unix (operating system) notes, but you should make yourself familiar with the basic concepts.
We will generally use the jupyter notebooks for running interactive Python programs.
The repository of these notes is https://github.com/profLewis/newform0111.
If you navigate to that link, and look under the Code tab, you will see various ways you can download the notes.
The main one we will uses git. You should first check that you have git installed (e.g. type git in a terminal / shell window).
We also assume you are using the conda distribution of python. Check that you have that installed (e.g. type conda in a terminal / shell window).
If either of these fail, then find the approrpate downloads to install them for your computer, and try again.
Change directory to where you want to work, e.g. (BUT put in your own directory name!):
cd /Users/Documents/GitHub
Make a local clone of the notes:
git clone https://github.com/profLewis/newform0111.git
Install this with:
python setup.py install && bash postBuild
Users can install the course utility codes with:
conda install newform0111
They can then access utility codes from this:
python -c "import newform0111;help(newform0111)"
Assuming we are using conda, and that you have an anaconda cloud account (https://anaconda.org). The account associated with this project is profLewis, so we use the account https://anaconda.org/proflewis/ as our example. Assume we are using conda for packages. We follow the anaconda instructions.
Package configuration is done in the file meta.yaml.
Assume we have the distribution locally in /Users/Documents/GitHub/newform0111:
cd /Users/Documents/GitHub/newform0111
# installs
conda install anaconda-client conda-build conda-verify
# To build the package, turn off automatic Client uploading and then run the conda build command:
# All packages built in this way are placed in a subdirectory of the Anaconda conda-bld directory.
conda config --set anaconda_upload no
conda build .
# You can check where the resulting file was placed with the --output option:
conda build . --output
# You can upload the test package to Cloud with the Anaconda upload command:
anaconda login
anaconda upload /path/to/conda-package.tar.bz2
The user can now access the package from this channel. They can set this up with:
conda config --add channels profLewis
Then install with:
conda install newform0111
They can then access utility codes from this:
python -c "import newform0111;help(newform0111)"
You can set up a docker machine by simply downloading the Dockerfile file.
Make sure you have docker installed on your local machine.
Then cd to where you stored the Dockerfile and run:
docker build --rm -t jupyter/geog0111 .
This will take some time to download and install (10s of minutes), but when it is:
docker run -it --rm -p 8888:8888 jupyter/geog0111 start.sh jupyter lab
Then, to link to (my) docker space (use your own docker usename in place of proflewis):
docker tag jupyter/geog0111 proflewis/geog0111
Upload:
docker push proflewis/geog0111
This now exists as https://hub.docker.com/repository/docker/proflewis/geog0111, and you can now simply load it as:
docker pull proflewis/geog0111:latest
To run commands on the docker image as root, for example install:
docker run -u 0 -it -p 8888:8888 proflewis/geog0111:latest bash -c "cd newform0111 && python setup.py install && bash postBuild && conda activate newform0111"
Additional info:
To clear cache and force a reload of the repository, use:
docker system prune -af
Then
docker build --rm -t jupyter/geog0111 .
etc. as above
To update notes on docker image:
docker run -u 0 -w /home/jovyan/newform0111 -it proflewis/geog0111:latest bash -c "git reset --hard HEAD && git pull"
docker run -u 0 -w /home/jovyan/newform0111 -it proflewis/geog0111:latest bash -c "python setup.py install && bash postBuild && conda activate newform0111"
Make sure you install docker on you local computer, then type:
docker pull proflewis/geog0111:latest
Then, run it with:
docker run -it -w /home/jovyan/newform0111 -p 8888:8888 proflewis/geog0111:latest start.sh jupyter notebook