Utilizing spaCy with Transformers
Transformers are the latest hot topic in NLP. The goal of this chapter is to learn how to use transformers to improve the performance of trainable components in spaCy.
First, you will learn about transformers and transfer learning. Next, you’ll learn more about spaCy trainable components and how to train a component, introducing spaCy’s config.cfg files and spaCy’s CLI. Then, you will learn about the architectural details of the commonly used Transformer architecture – Bidirectional Encoder Representations from Transformers (BERT) and its successor, RoBERTa. Finally, you’ll train the TextCategorizer component to classify texts using a transformer layer to improve accuracy.
By the end of this chapter, you will be able to prepare data for training and fine-tune your own spaCy components. Because of the way spaCy is designed; while doing that, you’ll be following software engineering best practices. You...
