Module 10
Natural language processing
Tokenisation with NLTK, regular expressions, bag of words, word2vec, sentiment analysis, Markov chains and a first chatbot.
Outcome
What you will be able to do
You can turn raw text into features and build a working sentiment classifier end to end.
Lessons
Work through these in order
- 01What is Natural Language ProcessingAs we said before, data scientists with NLP skills are high in demand in the industry. That's basically because there are many real-world applications that somehow involve NLP tasks. Here we mention… 8 min
- 02NLTK - Natural Language ToolKitTokenization is a method of breaking up a piece of text into many pieces, such as sentences and words, and is an essential first step for recipes in the later chapters. 8 min
- 03MetaCharactersSquare brackets specify a set of characters (a character class) you wish to match. All characters written between these square brackets are taken into account. Gr[ae]y can match both gray and grey. 8 min
- 04Bag of WordsTo create a Count Vectorizer, we simply need to instantiate one. We are not using any parameters yet. 8 min
- 05Supervised/Unsupervised NLP ExampleX_train, X_test, y_train, y_test = train_test_split(X, 8 min
- 06Intro to word2vecIn considering the relationship between a word and its surrounding words, word2vec has two options that are the inverse of one another: 8 min
- 07Sentiment Analysis Movie Reviewtype of text train: <class 'list'> length of text_train: 25000 text_train[1]: b"Zero Day leads you to think, even re-think why two boys/young men would do what they did - commit mutual suicide via… 8 min
- 08What is Markov chain?In the following examples, we'll use a library called markovify to generate the Markov chains using Jane Austen's novel Emma as our corpus. You can install markovify from the terminal (or command… 8 min
- 09ChatbotBefore moving on to the implementation, let's talk a little bit about chatbots. In a nutshell: 8 min
Assessment
Module quiz — 70% to pass
6 questions mixing concept checks and short code-output problems. Graded on the server with per-question explanations afterwards, unlimited retakes, and a badge with a verification code the moment you pass.