2110446 Data Science and Data Engineering @Chula (2022/2)

Syllabus:

Syllabus

Registration request (expired)

Course material:

Week01: Intro to Numpy, Pandas

1. Numpy:

2. Pandas:

3. Pandas with Youtube stat data:

4. (Advanced) Pandas with Youtube stat data:

Assignment (Pandas with Youtube stat data):

Week02: Data Preparation

1. EDA:

2. Impute Missing Value:

3. Split Train/Test:

4. Outliers with Log:

5. Outliers with Log (Titanic DataSet):

Assignment:

Week03-04: Traditional ML

1. Decision Trees:

2. Linear Regression:

3. Logistic Regression:

4. Neural Network:

5. K Nearest Neighbors:

6. SVM:

7. Save and Load Model:

8. K-Means:

9. Market-Basket Analysis:

Week05-06: Intro to Deep Learning

1. Image classification (basic): CIFAR10

2. Image classification (advanced): Animal

3. Object detection: VOCDetection

4. Semantic segmentation: CamSeq2007

5. Time series Forecasting: Stock Price

Assignment:

Week08: Introduction to Data Science and Big Data Architecture

1. Simple example

Assignment:

Week09: Data Extraction

1. Basic Webpage Scraping -- Note: File 'simple_page.html' will be uploaded to Colab automatically.

2. Wikipedia page data extraction

3. REST API Data Extraction

4. Twitter Data Extraction -- Note: Do not forget to upload "twitter.yml" to your colab machine and modify the bearer token value.

5. Selenium -- Note: this example cannot be run on Colab.

Assignment:

Week10: Data Extraction

This section contains example for Kafka. To test, you can use Kafka Server using IP 35.240.149.229 port 9092 or local server.

To run local server, install kafka locally or use the following docker compose file

Basic Example:

• Producer

• Consumer

• Consumer 2

• Producer 2

Complex Example

• Sensor A Producer

• Sensor B Producer

• File Writer Consumer

• Counter Consumer

• Notifier Consumer

AVRO Example

• AVRO Producer

• AVRO Consumer

• Sample AVRO Schema sample.avsc

Consumer Group Example

• Partition Producer

• Consumer Group 1

• Consumer Group 2

• Consumer Group 3

Assignment

• Transaction Verifier

• transaction.avsc

• submit.avsc

• result.avsc