Module 07
Data exploration and feature engineering
Variable types, missing values, outliers, univariate and bivariate analysis, transformations, feature selection and PCA — the longest course in the bootcamp.
Outcome
What you will be able to do
You can take a raw table and produce a modelling-ready feature set you can justify decision by decision.
Lessons
Work through these in order
- 01Handling with Variable TypeIn this part of the curriculum, we will discuss main tools that we can use in data cleansing. Before proceeding, please keep in mind that tools that we are about to learn are not all-inclusive. As we… 8 min
- 02Detecting missing valuesBy eyeballing, it is easy to notice whether there is a missing value in Python. If you encounter a value of NAN(Not a Number) in the dataset, a ring should bell and say: Here you go, you have a… 8 min
- 03Why should outliers be treated with caution?Some immediately treat outliers as something we need to delete but this is something that a data scientist should stay away. In machine learning, model are data dependent implying that it would not… 8 min
- 04Statistical Tools in Detecting the OutliersThe Z-Score is a scaling technique by which we have a dataset with mean 0 and standard deviation of 1. So, z-score converts a data considered to be normally distributed into the standard normal… 8 min
- 05Fixing the OutliersNow, we know how to detect outliers and we are good at it but it is not the final step. Pretend that we detect the outliers, it is now time to make a decision about how to deal with them. In order to… 8 min
- 06Univariate AnalysisThe term Exploratory Data Analysis (EDA) was first coined by John Tukey. This is a process in which we understand the data, check the shape and apply some visualizations, make decision about… 8 min
- 07CorrelationAs you may notice there are many ways to find the interaction between variables. However, it is no surprise that finding out the correlation coefficient is the best and easiest way. In Python, there… 8 min
- 08Transforming the FeaturesSince machine learning models work only on numerical data, we should convert our categorical data to a numerical value. One method for this purpose is called one-hot encoding. In one-hot encoding, we… 8 min
- 09Principal Components AnalysisPCA works best for normally distributed data and assumes the relationships among variables are linear. PCA also works best when the variables involved range from weak to moderately strong… 8 min
Assessment
Module quiz — 70% to pass
6 questions mixing concept checks and short code-output problems. Graded on the server with per-question explanations afterwards, unlimited retakes, and a badge with a verification code the moment you pass.