Welcome to DSCI 571: Supervised Learning I! This course introduces fundamental concepts and techniques in supervised machine learning such as data splitting, cross-validation, generalization, overfitting, the fundamental trade-off, the golden rule, and data preprocessing. Additionally, we will explore popular machine learning algorithms, including decision trees,
Click to expand!
By the end of this course, you will be able to:
- Describe supervised learning and its suitability for various tasks.
- Explain key machine learning concepts such as classification, regression, overfitting, and the trade-off in model complexity.
- Identify appropriate data preprocessing techniques for specific scenarios, provide reasons for their selection, and integrate them into machine learning pipelines.
- Develop an intuitive understanding of common machine learning algorithms.
- Build end-to-end supervised machine learning pipelines using Python and scikit-learn on real-world datasets.
Click to expand!
The following deliverables will determine your course grade:
| Assessment | Weight | Where to submit |
|---|---|---|
| Lab Assignment 1 | 12% | Gradescope |
| Lab Assignment 2 | 12% | Gradescope |
| Lab Assignment 3 | 12% | Gradescope |
| Lab Assignment 4 | 12% | Gradescope |
| iClicker participation | 2% | iClicker Cloud |
| Quiz 1 | 25% | Canvas |
| Quiz 2 | 25% | Canvas |
See Calendar for the due dates.
Click to expand!
| Role | Name |
|---|---|
| Lecture Instructor | Varada Kolhatkar |
| Lab Instructor | Varada Kolhatkar |
| Teaching Assistant | Armin Saadat Boroujeni |
| Teaching Assistant | Daniel Ramandi |
| Teaching Assistant | Faeze Keshavarz |
| Teaching Assistant | Md Shahriar Rahim Siddiqui |
| Teaching Assistant | Negar Sadrzadeh |
| Teaching Assistant | Prajeet Bajpai |
Click to expand!
This course follows a semi-flipped classroom format, where you will watch pre-recorded videos before class. In-class sessions will focus on demos, iClicker questions, Q&A, discussions, and worksheets. It's optional but highly recommended to download the appropriate datasets provided below and put them under your local lectures/data directory, and run the lecture Jupyter notebooks on your own and experiment with the code.
This course occurs during Block 2 in the 2023/24 school year.
| Lecture | Topic | Assigned videos | Resources and optional readings |
|---|---|---|---|
| Course information | 📹 Pre-watch: 1.0 | ||
| 1 | Terminology, baselines, decision trees | 📹 Pre-watch: 2.1, 2.2, 2.3, 2.4 | |
| 2 | ML fundamentals | 📹 Pre-watch: 3.1, 3.2, 3.3, 3.4 | An article by Pedro Domingos |
| 3 | $k$-NNs, SVM RBF | 📹 Pre-watch: 4.1, 4.2, 4.3, 4.4, 5.1 | |
| 4 | Preprocessing, pipelines, column transformer | 📹 Pre-watch: 5.2, 5.3, 5.4, 6.1 | |
| 5 | More preprocessing, text features | 📹 Pre-watch: 6.2 | |
| 6 | Hyperparameter optimization, optimization bias | 📹 Pre-watch: 8.1, 8.2 | |
| 7 | Naive Bayes | None |
|
| 8 | Logistic Regression, multi-class classification | 📹 Pre-watch: 7.1, 7.2, 7.3 |
Here is the list of Kaggle datasets we'll use in this class.
- SMS Spam Collection Dataset
- Indian Liver Patient Records
- IMDB movie reviews
- House Sales in King County
- The adult census dataset
- Spotify Song Attributes
- California Housing
If you want to be extra prepared, you may want to download these datasets in advance and save them under the lectures/data directory in your local copy of the repository.
During labs, you will be given time to work on your own or in groups. There will be a lot of opportunity for discussion and getting help during lab sessions. (Usually I enjoy labs a lot. It's also an opportunity for me to know you a bit better 🙂.)
We are providing you with a conda environment file which is available here. You can download this file and create a conda environment for the course and activate it as follows.
conda env create -f env-dsci-571.yaml
conda activate 571
In order to use this environment in Jupyter, you will have to install nb_conda_kernels in the environment where you have installed Jupyter (typically the base environment). You will then be able to select this new environment in Jupyter. If you're unable to see the environment in Jupyter, you might have to install the kernel manually. See the documentation here. For more details on this, refer to your 521 lecture 7.
I've only attempted to install this environment file on a few machines, and you may encounter issues with certain packages from the yaml file when executing the commands above. This is not uncommon and may suggest that the specified package version is not yet available for your operating system via conda. When this occurs, you have a couple of options:
- Modify the local version of the
yamlfile to remove the line containing that package. - Create the environment without that package.
- Activate the environment and install the package manually either with
conda installorpip installin the environment.
Note that this is not a complete list of the packages we'll be using in the course and there might be a few packages you will be installing using conda install later in the course. But this is a good enough list to get you started.
Click to expand!
We are all here to support your learning and success in the course and the program. Here's how our communication will work during the course.
If there is any clarification on the lecture material or lab questions, I'll open an issue in the course repository and tag you. I will also post a Slack message and tag you. It is your responsibility to read the messages whenever you are tagged. (I know that there are too many things for you to keep track of. You do not have to read all the messages but please make sure to carefully read the messages whenever you are tagged.)
If you have questions about the lecture material or lab questions please post them on the course Slack channel rather than direct messaging me or the TAs. Here are the advantages of doing so:
- You'll get a quicker response.
- Your classmates will benefit from the discussion.
When you ask your question on the course channel, please avoid tagging the instructor unless it's specific for the instructor (e.g., if you notice some mistake in the lecture notes). If you tag a specific person, other teaching team members or your colleagues are discouraged to respond. This decreases the response rate on the channel.
Please use some consistent convention when you ask questions on Slack to facilitate easy search for others or future you. For example, if you want to ask a question on Exercise 3.2 from Lab 1, start your post with the label lab1-ex2.3. Or if you have a question on lecture 2 material, start your post with the label lecture2. Once the question is answered/solved, you can add "(solved)" tag before the label (e.g., (solved) lab1-ex2.3. Do not delete your post even if you figure out the answer on your own. The question and the discussion can still be beneficial to others.
If you have concerns related to grading:
- First, ensure that you have thoroughly reviewed your response, our solution key, and any TA feedback.
- Verify that your concerns align with our 'Reasonable grading concerns' policy, which you can read here.
If you believe your concerns are valid:
- For assignments: Submit a regrade request on Gradescope.
- For quizzes: Directly message the TA responsible for grading that question. (I will inform you on Slack who has graded what after grades are returned.)
- If you cannot resolve the issue with the TA, send a Slack message to the instructor, including the relevant TA in the conversation.
I am open to having a conversation with you. If you'd like to discuss any sensitive matters, please send me a direct message on Slack (and mention/tag me) instead of posting it in the course channel. It may take some time for me to respond, but I'll make every effort to reply as promptly as I can. If I cannot address your issue directly, I can at least direct you to the appropriate person who may be able to assist you.
Click to expand!
- A Course in Machine Learning (CIML) by Hal Daumé III (also relevant for DSCI 572, 573, 575, 563)
- Introduction to Machine Learning with Python: A Guide for Data Scientists by Andreas C. Mueller and Sarah Guido.
- An Introduction to Statistical Learning
- The Elements of Statistical Learning (ESL)
- Data Mining: Practical Machine Learning Tools and Techniques (PMLTT)
- Artificial intelligence: A Modern Approach by Russell, Stuart and Peter Norvig.
- Artificial Intelligence 2E: Foundations of Computational Agents (2017) by David Poole and Alan Mackworth (of UBC!).
- CPSC 330
I'm currently teaching an undergrad course on applied machine learning. Unlike DSCI 571, CPSC 330 is a semester-long course but there is a lot of overlap and sharing of notes between these courses. - Machine Learning Crash Course
- Mike's CPSC 340
- Machine Learning (Andrew Ng's famous Coursera course)
- Foundations of Machine Learning online course from Bloomberg.
- Machine Learning Exercises In Python, Part 1 (translation of Andrew Ng's course to Python, also relevant for DSCI 561, 572, 563)
- A Visual Introduction to Machine Learning (Part 1)
- A Few Useful Things to Know About Machine Learning (an article by Pedro Domingos)
- Metacademy (sort of like a concept map for machine learning, with suggested resources)
- Machine Learning 101 (slides by Jason Mayes, engineer at Google)
Please see the general MDS policies.
Enjoy your learning journey in DSCI 571: Supervised Learning I!