Summary
In this Learn Module, we have covered all the basics of Natural Language Processing such as: text representation, traditional recurrent network models, and near state-of-the-art models with attention. We were focusing mostly on text classification and did not discuss in detail other important tasks like named entity recognition, machine translation and question answering. To implement those tasks, the same basic RNN principles are used with a different top layer architecture. To get a more complete understanding of the NLP field, you should experiment with some of those problems as well.
One of the other emerging areas of NLP is model visualization and probing. This direction is also known as BERTology. As we have seen in the previous unit, visualizing attention matrix can tell us a lot about how machine translation works and where the model "looks" when translating a word. There are other powerful methods of understanding BERT internals.
The latest text generative models, such as GPT-2/3, are slightly different from BERT, in a sense that they can be "programmed" to solve different tasks just by providing an "initial sequence" for text generation. This could lead to a possible paradigm shift where instead of doing transfer learning training we would be focusing on creating suitable questions for giant pre-trained networks. If you want to get really serious about NLP, you probably need to explore some of the latest text generative models, such as GPT-2, or Microsoft Turing NLG.
Now you have the basics to get started on any natural language task!