Scribd
Upload
452 views2 pages

Contoh Soal N Gram (Bagus)

This document contains exercises on using n-gram language models for natural language processing. It provides sample training data and asks students to calculate n-gram probabilities, predict next words, compute perplexity, and determine the most probable sentences given a bigram language model with and without Laplace smoothing. Solutions are provided that calculate the relevant probabilities and perplexities.

Uploaded by

yeninur
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
452 views2 pages

Contoh Soal N Gram (Bagus)

This document contains exercises on using n-gram language models for natural language processing. It provides sample training data and asks students to calculate n-gram probabilities, predict next words, compute perplexity, and determine the most probable sentences given a bigram language model with and without Laplace smoothing. Solutions are provided that calculate the relevant probabilities and perplexities.

Uploaded by

yeninur
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Machine Learning

Exercises: language models (n-grams)


Laura Kallmeyer

Summer 2016, Heinrich-Heine-Universität Düsseldorf

Exercise 1 Consider the following toy example (similar to the one from Jurafsky & Martin (2015)):
Training data:
<s> I am Sam </s>
<s> Sam I am </s>
<s> Sam I like </s>
<s> Sam I do like </s>
<s> do I like Sam </s>

Assume that we use a bigram language model based on the above training data.

1. What is the most probable next word predicted by the model for the following word sequences?

(1) <s> Sam . . .


(2) <s> Sam I do . . .
(3) <s> Sam I am Sam . . .
(4) <s> do I like . . .

2. Which of the following sentences is better, i.e., gets a higher probability with this model?

(5) <s> Sam I do I like </s>


(6) <s> Sam I am </s>
(7) <s> I do like Sam I am </s>

Solution:
Bigram probabilities:
P (Sam|<s>) = 35 P (I|<s>) = 15
P (I|Sam) = 53 P (</s>|Sam) = 25
P (Sam|am) = 12 P (</s>|am) = 21
P (am|I) = 25 P (like|I) = 52 P (do|I) = 1
5
P (Sam|like) = 31 P (</s>|like) = 23
P (like|do) = 12 P (I|do) = 12

1. (1) and (3): “I”.


(2): “I” and “like” are equally probable.
(4): </s>
2. Probabilities:
3 3 1 1 2 2
(5): 5 · 5 · 5 · 2 · 5 · 3
3 3 2 1
(6): 5 · 5 · 5 · 2
1 1 1 1 3 2 1
(7): 5 · 5 · 2 · 3 · 5 · 5 · 2
(6) is the most probable sentence according to our language model.
Exercise 2 Consider again the same training data and the same bigram model. Compute the perplexity
of

<s> I do like Sam

Solution:
1 1 1 1 1
The probability of this sequence is 5 · 5 · 2 · 3 = 150 .
√
The perplexity is then 4 150 = 3.5

Exercise 3 Take again the same training data. This time, we use a bigram LM with Laplace smoothing.

1. Give the following bigram probabilities estimated by this model:


P (do|<s>) P (do|Sam) P (Sam|<s>) P (Sam|do)
P (I|Sam) P (I|do) P (like|I)
Note that for each word wn−1 , we count an additional bigram for each possible continuation wn .
Consequently, we have to take the words into consideration and also the symbol </s>.
2. Calculate the probabilities of the following sequences according to this model:

(8) <s> do Sam I like


(9) <s> Sam do I like

Which of the two sequences is more probable according to our LM?

Solution:

1. If we include </s> (this can also appear as second element of a bigram), we get |V | = 6 for our
vocabulary.
2 1 4 1
P (do|<s>) = 11 P (do|Sam) = 11 P (Sam|<s>) = 11 P (Sam|do) = 8
4
P (I|Sam) = 11 P (I|do) = 82 3
P (like|I) = 11
2 1 4 3
2. (8): 11 · 8 · 11 · 11
4 1 2 3
(9): 11 · 11 · 8 · 11
The two sequences are equally probable.

References
Jurafsky, Daniel & James H. Martin. 2015. Speech and language processing. an introduction to natural language processing,
computational linguistics, and speech recognition. Draft of the 3rd edition.

You might also like