Partitioning text into sentences is also called sentence boundary disambiguation (SBD). This process is useful for many downstream NLP tasks that require analysis within sentences; for example, POS and phrase analysis typically work within a sentence.
In this chapter, we will explain why SBD is difficult. Then, we will examine some core Java approaches that may work in some situations, and move on to the use of models by various NLP APIs. We will also examine training and validating approaches for sentence-detection models. We can add additional rules to refine the process further, but this will work only up to a certain point. After that, models must be trained to handle both common and specialized situations. The latter part of this chapter focuses on these models and their use.
We will cover the following topics in this chapter:
- The SBD process
- What makes...
