3174205 211310142028

Natural Language Processing

Name:Jal Patel

Enrollment No: 211310142028

Branch: CSE(AI-ML)

Batch: A4

Semester: 7th

1
 3174205 211310142028

LAB MANUAL
Sr. Practical Name Date of Faculty
No. Submission Signature
1 Write a program to implement word Tokenizer, Sentence and
Paragraph Tokenizers
2 Write a python program to eliminate stopwords using NLTK.
3 Write a python program to perform Parts of Speech tagging
using NLTK.
4 Write a python program to perform lemmatization using
NLTK.
5 Write a python program for chunking using NLTK.
6 Write a python program to perform stemming using NLTK.
7 Write a python program to perform Named Entity Recognition
using NLTK.
8 Write a program for different feature extraction techniques
used in NLP.
9 Write a program to implement all the NLP Pre-Processing
Techniques required to perform further NLP tasks.
10 Write a program to implement both user-defined and pre-
defined functions to generate: Unigrams, Bigrams, Trigrams, N-
grams.
11 Write a program to identify all antonyms and synonyms of a
word.
12 Write a program to find hyponymy, homonymy, polysemy for a
given word.
13 Write a program to calculate the score and polarity of text data using
vader analyzer and TextBlob.

2
 3174205 211310142028

Practical 1
Write a program to implement word Tokenizer, Sentence and Paragraph
Tokenizers

• Tokenization:

Tokenization is one of the fundamental steps in Natural Language Processing (NLP). It

involves splitting text into smaller components such as sentences, paragraphs, or words,
making the data easier to process. Tokenization helps in the pre-processing stage, where text is
broken down for further analysis. There are three common types of tokenizers:

1. Word Tokenizer: Breaks down text into individual words.

2. Sentence Tokenizer: Splits text into sentences.
3. Paragraph Tokenizer: Splits large text blocks into paragraphs.

A. Sentence Tokenization
Sentence tokenization refers to splitting a given text into individual sentences. This is useful
for analysing sentence structures and is a typical first step in NLP applications like sentiment
analysis, text summarization, and translation. Python's nltk library provides a simple function
sent_tokenize() for this task.

Code:
# Step 1: Import the necessary libraries
import nltk
nltk.download('punkt') # Download necessary tokenizer models
from nltk.tokenize import sent_tokenize

# Step 2: Define the sentence tokenization function

def tokenize_sentences(text):
sentences = sent_tokenize(text) # Split text into sentences
return sentences

# Step 3: Input paragraph for tokenization

text = """Natural language processing (NLP) is a machine learning technology
that gives computers the ability to interpret, manipulate, and comprehend human language.
Organizations today have large volumes of voice and text data from various communication channels
like emails, text messages,
social media newsfeeds, video, audio, and more. They use NLP software to automatically process this
data,
analyze the intent or sentiment in the message, and respond in real time to human communication."""

# Step 4: Tokenize sentences

sentences = tokenize_sentences(text)

# Step 5: Print tokenized sentences

3
 3174205 211310142028

for i, sentence in enumerate(sentences):

print(f"Sentence {i+1}: {sentence}")

Output:

B. Word Tokenization
Word tokenization splits a text into individual words, which is essential for tasks like frequency
analysis, text classification, and language modeling. nltk provides the word_tokenize()
function for this.

Code:
# Step 1: Import necessary libraries
import nltk
nltk.download('punkt') # Download tokenizer models
from nltk.tokenize import word_tokenize

# Step 2: Define the word tokenization function

def tokenize_words(text):
words = word_tokenize(text) # Split text into words
return words

# Step 3: Input text for tokenization

text = "Natural language processing (NLP) is a machine learning technology that gives computers the
ability to interpret, manipulate, and comprehend human language."

# Step 4: Tokenize words

words = tokenize_words(text)

# Step 5: Print tokenized words

print(words)

Output:

C. Paragraph Tokenization
Paragraph tokenization involves dividing large bodies of text into separate paragraphs. This
method is useful in organizing large documents for structured analysis. Though there is no
direct function in nltk for paragraph tokenization, we can achieve this by splitting the text using
newlines (\n).
4
 3174205 211310142028

Code:
# Step 1: Define the paragraph tokenization function
def tokenize_paragraphs(text):
paragraphs = text.split('\n\n') # Split text by newlines
return paragraphs

# Step 2: Input text with paragraphs

text = """Natural language processing (NLP) is a machine learning technology that gives computers the
ability to interpret, manipulate, and comprehend human language.

Organizations today have large volumes of voice and text data from various communication channels
like emails, text messages, social media newsfeeds, video, audio, and more.

They use NLP software to automatically process this data, analyze the intent or sentiment in the
message, and respond in real time to human communication."""

# Step 3: Tokenize paragraphs

paragraphs = tokenize_paragraphs(text)

# Step 4: Print tokenized paragraphs

for i, paragraph in enumerate(paragraphs):
print(f"Paragraph {i+1}: {paragraph}")

Output:

5
 3174205 211310142028

Practical 2
Write a python program to eliminate stopwords using NLTK

• Introduction

Stopwords are common words in a language (like "the", "is", "in", etc.) that often do not add
significant meaning to a sentence and can be ignored during text analysis. In Natural Language
Processing (NLP), removing stopwords is an important preprocessing step to focus on
meaningful words that contribute to the context of the text.

Python's Natural Language Toolkit (NLTK) provides a ready-to-use list of stopwords for
different languages. By using the stopwords module, we can easily filter out these words from
the input text.

Code:
# Step 1: Import necessary libraries
import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize

# Step 2: Download NLTK stopwords and tokenizer models

nltk.download('stopwords') # Stopwords for various languages
nltk.download('punkt') # Tokenizer model for word tokenization

# Step 3: Define a function to remove stopwords

def remove_stopwords(text):
# Tokenize the text into words
words = word_tokenize(text)

# Get English stopwords from NLTK

english_stopwords = set(stopwords.words('english'))

# Filter out stopwords from the tokenized words

filtered_words = [word for word in words if word.lower() not in english_stopwords]

# Join the filtered words back into a single string

filtered_text = ' '.join(filtered_words)

return filtered_text

# Step 4: Example text for demonstration

text = "NLTK is a leading platform for building Python programs to work with human language data."

# Step 5: Remove stopwords from the example text

filtered_text = remove_stopwords(text)

# Step 6: Print the text after stopword removal

print("Original Text: ", text)

6
 3174205 211310142028

print("Filtered Text (Without Stopwords): ", filtered_text

Output:

7
 3174205 211310142028

Practical 3
Write a python program to perform Parts of Speech tagging using NLTK.

• Introduction
Parts of Speech (POS) tagging is a technique in Natural Language Processing (NLP) that
assigns a specific part of speech (noun, verb, adjective, etc.) to each word in a sentence. This
helps in understanding the syntactic structure of a sentence. The Natural Language Toolkit
(NLTK) in Python provides a convenient method for performing POS tagging using the
pos_tag() function.

A. POS Tags Overview

Some common POS tags used are:
• NN: Noun
• VB: Verb
• JJ: Adjective
• RB: Adverb

Code:
# Step 1: Import necessary libraries
import nltk
from nltk.tokenize import word_tokenize

# Step 2: Download NLTK tokenizer and POS tagging models

nltk.download('punkt') # Tokenizer model for word tokenization
nltk.download('averaged_perceptron_tagger') # POS tagging model

# Step 3: Define a function to perform POS tagging

def pos_tagging(text):
# Tokenize the text into words
words = word_tokenize(text)

# Perform POS tagging

tagged_words = nltk.pos_tag(words)

return tagged_words

# Step 4: Example text for POS tagging

text = "NLTK is a leading platform for building Python programs to work with human language data."

# Step 5: Perform POS tagging

tagged_text = pos_tagging(text)

# Step 6: Print the POS tagged text

print("POS Tagged Text:")
for word, tag in tagged_text:
print(f"{word} : {tag}")

8
 3174205 211310142028

Output:

9
 3174205 211310142028

Practical 4
Write a python program to perform lemmatization using NLTK

• Introduction
Lemmatization is a process in Natural Language Processing (NLP) where words are reduced
to their base or dictionary form, known as the lemma. Unlike stemming, lemmatization takes
into account the morphological analysis of words, ensuring that the base form is a valid word
in the language. For example, "running" becomes "run", and "better" becomes "good". This
helps in standardizing words to improve text analysis and retrieval.

NLTK provides the WordNet Lemmatizer, which is used for lemmatization based on WordNet,
a lexical database of the English language.

Code:
# Step 1: Import necessary libraries
from nltk.tokenize import word_tokenize
from nltk.stem import WordNetLemmatizer
import nltk
import os

# Step 2: Download necessary NLTK data (wordnet and tokenizer models)

nltk.download('punkt') # Tokenizer model for word tokenization
nltk.download('wordnet') # WordNet model for lemmatization

# Step 3: Find and print the path of the downloaded wordnet resource
try:
wordnet_path = nltk.data.find('corpora/wordnet.zip')
print(f"WordNet is located at: {wordnet_path}")
except LookupError:
print("WordNet not found, please make sure it's downloaded correctly.")

# Step 4: Define a function to perform lemmatization

def lemmatize_text(text):
# Create a WordNetLemmatizer object
lemmatizer = WordNetLemmatizer()

# Tokenize the text into words

tokens = word_tokenize(text)

# Lemmatize each word and join them back into a sentence

lemmatized_text = ' '.join([lemmatizer.lemmatize(word) for word in tokens])

return lemmatized_text

# Step 5: Example text for lemmatization

text = "The cats are chasing mice and playing in the garden"

# Step 6: Perform lemmatization on the example text

10
 3174205 211310142028

lemmatized_text = lemmatize_text(text)

# Step 7: Print the original and lemmatized text

print("Original Text:", text)
print("Lemmatized Text:", lemmatized_text)

Output:

11
 3174205 211310142028

Practical 5
Write a python program for chunking using NLTK

• Introduction
Chunking, also known as shallow parsing, is a process in Natural Language Processing (NLP)
that divides a text into syntactically correlated parts like noun phrases (NP), verb phrases (VP),
and more. The primary goal of chunking is to label segments of the sentence and group them
together based on their syntactical roles. It can be considered a higher-level task than
tokenization or part-of-speech (POS) tagging. Chunking provides us with more meaningful
groups of words that help in understanding the structure of a sentence.
Chunking is useful in various NLP tasks, such as:

• Named Entity Recognition (NER)

• Information Extraction
• Syntactic Analysis

• Steps in Chunking:
1. Tokenization: Splitting the sentence into individual words.
2. POS Tagging: Assigning part-of-speech tags (like noun, verb, adjective) to each token.
3. Chunking: Grouping words into meaningful phrases such as noun phrases (NP), verb
phrases (VP), etc.

Code:
import nltk
from nltk.tokenize import word_tokenize
from nltk import pos_tag, RegexpParser

# Download necessary NLTK data

nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')
def chunk_sentence(sentence):
# Tokenize the sentence into words
words = word_tokenize(sentence)

# Assign part-of-speech tags to each word

tagged_words = pos_tag(words)

# Define grammar for chunking using regular expressions

grammar = r"""
NP: {<DT|JJ|NN.*>+} # Chunk sequences of DT (determiners), JJ (adjectives), NN (nouns)
PP: {<IN><NP>} # Chunk prepositions followed by a noun phrase (NP)
VP: {<VB.*><NP|PP|CLAUSE>+$} # Chunk verb phrases (VP) with their arguments (NP or PP)
CLAUSE: {<NP><VP>} # Chunk a clause, consisting of a noun phrase (NP) followed by a verb
phrase (VP)
"""

# Create a chunk parser with the defined grammar

parser = RegexpParser(grammar)

12
 3174205 211310142028

# Parse the tagged words into chunks

chunked_sentence = parser.parse(tagged_words)

return chunked_sentence
# Example sentence
sentence = "The quick brown fox jumps over the lazy dog"

# Perform chunking on the sentence

chunked_sentence = chunk_sentence(sentence)

# Print the chunked sentence

print(chunked_sentence)

Output:

13
 3174205 211310142028

Practical 6
Write a python program to perform stemming using NLTK.

• Introduction

Stemming is a text normalization technique in Natural Language Processing (NLP) used to

reduce words to their root form. Unlike lemmatization, which returns valid dictionary words,
stemming often results in words that may not be real words but still capture the core meaning.

The Porter Stemmer, developed by Martin Porter, is one of the most widely used stemming
algorithms in NLP tasks. It applies a set of rules to remove common morphological and
inflexional endings from words in English.

Code:

import nltk
from nltk.stem import PorterStemmer
from nltk.tokenize import word_tokenize
# Download NLTK tokenizer
nltk.download('punkt')
def stem_text(text):
# Initialize the Porter Stemmer
porter_stemmer = PorterStemmer()
# Tokenize the text into words
words = word_tokenize(text)
# Apply stemming to each word
stemmed_words = [porter_stemmer.stem(word) for word in words]
# Join the stemmed words back into a single string
stemmed_text = ' '.join(stemmed_words)
return stemmed_text
# Example text
text = "NLTK is a leading platform for building Python programs to work with human language data."
# Perform stemming
stemmed_text = stem_text(text)
# Print the stemmed text
print(stemmed_text)

Output:

14
 3174205 211310142028

Practical 7
Write a python program to perform Named Entity Recognition using NLTK

• Introduction

Named Entity Recognition (NER) is a crucial task in Natural Language Processing (NLP),
where the goal is to locate and classify named entities in text into predefined categories such
as person names, organizations, locations, dates, and more.

The NLTK library in Python provides tools to perform NER using its built-in ne_chunk()
method, which uses part-of-speech tagged words to identify named entities.

Code:
import nltk
from nltk.tokenize import word_tokenize
from nltk import pos_tag, ne_chunk

# Download necessary NLTK resources

nltk.download('punkt') # Tokenizer
nltk.download('averaged_perceptron_tagger') # POS tagger
nltk.download('maxent_ne_chunker') # Named entity chunker
nltk.download('words') # Word list for NER
def ner(text):
# Tokenize the input text
words = word_tokenize(text)

# Tag the tokens with parts of speech

tagged_words = pos_tag(words)

# Perform Named Entity Recognition

named_entities = ne_chunk(tagged_words)

return named_entities
# Example text
text = "Apple is a company based in California, United States. Steve Jobs was one of its founders."

# Perform Named Entity Recognition

named_entities = ner(text)

# Print the identified named entities

print(named_entities)

15
 3174205 211310142028

Output:

16
 3174205 211310142028

Practical 8
Write a program for different feature extraction techniques used in NLP.

• Introduction
Feature extraction is a critical step in Natural Language Processing (NLP) that transforms raw
text into a structured format that can be easily analyzed by machine learning algorithms.
Several techniques can be used to extract features from text data, including Bag of Words,
Count Vectorization, TF-IDF, and Word2Vec.

A. Bag of Words is one of the simplest and most common techniques used in NLP for feature
extraction. It represents text data as a collection of words without considering the order of
words. Each unique word in the corpus is assigned a token, and the presence of a word in a
document is typically represented as a binary indicator (1 for present, 0 for absent) or as a count
of occurrences.

B. Count Vectorizer
Count Vectorizer is a specific implementation of the Bag of Words model that converts a
collection of text documents to a matrix of token counts. It counts the number of occurrences
of each word in the documents, resulting in a document-term matrix.

C. TF-IDF (Term Frequency-Inverse Document Frequency)

TF-IDF is an advanced feature extraction technique that evaluates how important a word is to
a document in a collection or corpus. It considers both the frequency of a word in a document
and the inverse frequency of that word across all documents. The formula for TF-IDF is:

TF-IDF(t,d)=(TF(t,d)×IDF(t))\(text{TF-IDF}(t, d) )= \text{TF}(t, d) \times \text{IDF}(t)TF-

IDF(t,d)=TF(t,d)×IDF(t)
where:
• TF(t,d)\text{TF}(t, d)TF(t,d) is the term frequency of term ttt in document ddd.
• IDF(t)=log⁡(NDF(t))\text{IDF}(t) = \log \left( \frac{N}{\text{DF}(t)}
\right)IDF(t)=log(DF(t)N) is the inverse document frequency of term ttt, where NNN
is the total number of documents and DF(t)\text{DF}(t)DF(t) is the number of
documents containing the term.

D. Word2Vec
Word2Vec is a more advanced technique that represents words as dense vectors in a continuous
vector space. Unlike the previous methods, Word2Vec captures semantic meaning and
relationships between words. It uses neural networks to learn word representations, allowing
words with similar meanings to be closer in the vector space.

Code for TF-IDF:

import nltk
import string
from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer
from gensim.models import Word2Vec
# Example Corpus for TF-IDF
corpus_tfidf = [

17
 3174205 211310142028

"We become what we think about",

"Happiness is not something readymade."
]

# Initialize TF-IDF Vectorizer

tfidf_vectorizer = TfidfVectorizer()

# Compute TF-IDF values

tfidf_matrix = tfidf_vectorizer.fit_transform(corpus_tfidf)

# Convert sparse matrix to numpy array to view

print("Vocabulary:", tfidf_vectorizer.vocabulary_)
print("IDF:", tfidf_vectorizer.idf_)
print("TF-IDF Vectors:", tfidf_matrix.toarray())

Output:

Code for Count Vectorizer:

# Example Corpus for Count Vectorizer

corpus_count = [
"We become what we think about",
"Happiness is not something readymade. It comes from your own actions"
]

# Initialize Count Vectorizer

count_vectorizer = CountVectorizer()

# Get counts of each token (word) in text data

count_matrix = count_vectorizer.fit_transform(corpus_count)

# Convert sparse matrix to numpy array to view

print("Vocabulary:", count_vectorizer.vocabulary_)
print("Shape:", count_matrix.shape)
print("Vectors:", count_matrix.toarray())

Output:

18
 3174205 211310142028

Code for Word2Vec:

# Example Corpus for Word2Vec
corpus_w2v = [
["We", "become", "what", "we", "think", "about"],
["Happiness", "is", "not", "something", "readymade", "it", "comes", "from", "your", "own", "actions"]
]

# Initialize Word2Vec Model

word2vec_model = Word2Vec(corpus_w2v, vector_size=100, window=5, min_count=1, workers=4)

# Get the vector for a word

word_vector = word2vec_model.wv['Happiness']
print("Word Vector for 'Happiness':", word_vector)
similar_words = word2vec_model.wv.most_similar('Happiness', topn=5)
print("Top 5 words similar to 'Happiness':", similar_words)

Output:

19
 3174205 211310142028

Practical 9
Write a program to implement all the NLP Pre-Processing Techniques
required to perform further NLP tasks.

• Introduction
NLP pre-processing is an essential step in preparing text data for further analysis or modeling.
Properly pre-processed data can lead to better model performance, as it helps reduce noise and
irrelevant information in the text. Different tasks in NLP may require specific pre-processing
techniques depending on the nature of the data and the analysis objectives.

Code:
# Import necessary libraries
import nltk
import string
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer, WordNetLemmatizer
from nltk import pos_tag, ne_chunk

# Download required NLTK resources

nltk.download('punkt')
nltk.download('stopwords')
nltk.download('averaged_perceptron_tagger')
nltk.download('maxent_ne_chunker')
nltk.download('words')

# Sample text data (sentiment analysis, healthcare, social media)

text_data = [
"I love the new healthcare policies! They're making a positive impact.",
"The service at the restaurant was terrible; I will never go back!",
"COVID-19 has changed our lives drastically. Stay safe and healthy!",
"My favorite place to relax is the beach; it calms my mind.",
"Social media can be toxic but also a great way to connect."
]

# Function to preprocess text

def preprocess_text(text):
# Initialize stemmer and lemmatizer
stemmer = PorterStemmer()
lemmatizer = WordNetLemmatizer()

# Tokenize text
tokens = word_tokenize(text)

# Convert to lowercase
tokens = [word.lower() for word in tokens]

# Remove punctuation

20
 3174205 211310142028

tokens = [word for word in tokens if word.isalnum()]

# Remove stop words

stop_words = set(stopwords.words('english'))
tokens = [word for word in tokens if word not in stop_words]

# Apply stemming
stemmed_tokens = [stemmer.stem(word) for word in tokens]

# Apply lemmatization
lemmatized_tokens = [lemmatizer.lemmatize(word) for word in stemmed_tokens]

return lemmatized tokens

# Function to perform Named Entity Recognition

def perform_ner(text):
words = word_tokenize(text)
tagged_words = pos_tag(words)
named_entities = ne_chunk(tagged_words)
return named_entities

# Preprocess each text data

for sentence in text_data:
preprocessed = preprocess_text(sentence)
print(f"Original Sentence: {sentence}")
print(f"Preprocessed Tokens: {preprocessed}")
print(f"Named Entities: {perform_ner(sentence)}")
print("\n" + "="*80 + "\n")

Output:

21
3174205 211310142028

22
 3174205 211310142028

Practical 10
Write a program to implement both user-defined and pre-defined functions
to generate: Unigrams, Bigrams, Trigrams, N-grams.

• Introduction:
In Natural Language Processing (NLP), N-grams are continuous sequences of 'n' items from a
given text or speech. These items can be words, letters, or syllables depending on the context
of application. N-grams play an important role in text preprocessing for tasks like text
generation, machine learning, sentiment analysis, and more.

• Uni-grams (1-grams): These are single words from the text.

• Bi-grams (2-grams): These are pairs of consecutive words.
• Tri-grams (3-grams): These are triplets of consecutive words.
• N-grams: General term for sequences of 'n' consecutive words.

• Theory:

• Uni-grams provide basic word-level analysis.

• Bi-grams and Tri-grams help in capturing adjacent word relationships or patterns in a
text, which can be important for understanding context and structure. They are
commonly used in building language models or understanding phrases.
• N-grams in general are used in predictive text models, auto-completion, and text
analysis applications. For larger N-values, N-grams can capture longer phrases, but as
N increases, data sparsity may become a challenge.

Code:
import nltk
from nltk.util import ngrams

# Downloading necessary NLTK resources

nltk.download('punkt')

# Sample text data

Text = 'this is a very good book to study'

# Pre-defined function using nltk to generate N-grams

def generate_ngrams_predefined(text, n):
"""
This function uses nltk's ngrams function to generate n-grams.

Parameters:
text: str - The input text to generate n-grams from
n: int - The number of n-grams (1 for uni-grams, 2 for bi-grams, etc.)

Returns:
list - List of n-grams

23
 3174205 211310142028

"""
tokens = nltk.word_tokenize(text)
return list(ngrams(tokens, n))

# User-defined function to generate N-grams

def generate_ngrams_userdefined(text, n):
"""
This function manually generates n-grams by iterating through the text.

Parameters:
text: str - The input text to generate n-grams from
n: int - The number of n-grams (1 for uni-grams, 2 for bi-grams, etc.)

Returns:
list - List of n-grams
"""
tokens = nltk.word_tokenize(text)
ngrams_list = []

for i in range(len(tokens) - n + 1):

ngram = tuple(tokens[i:i+n])
ngrams_list.append(ngram)

return ngrams_list

# Demonstration for Uni-grams, Bi-grams, and Tri-grams using Pre-defined and User-defined functions
for i in range(1, 4): # Loop for 1-gram, 2-grams, and 3-grams
print(f"\n{'='*20} {i}-grams (Pre-defined) {'='*20}")
ngrams_predefined = generate_ngrams_predefined(Text, i)
for grams in ngrams_predefined:
print(grams)

print(f"\n{'='*20} {i}-grams (User-defined) {'='*20}")

ngrams_userdefined = generate_ngrams_userdefined(Text, i)
for grams in ngrams_userdefined:
print(grams)

# Generate general N-grams (For example, 4-grams or 5-grams)

n=4
print(f"\n{'='*20} {n}-grams (Pre-defined) {'='*20}")
ngrams_predefined = generate_ngrams_predefined(Text, n)
for grams in ngrams_predefined:
print(grams)

print(f"\n{'='*20} {n}-grams (User-defined) {'='*20}")

ngrams_userdefined = generate_ngrams_userdefined(Text, n)
for grams in ngrams_userdefined:
print(grams)

24
 3174205 211310142028

Output:

25
 3174205 211310142028

Practical 11
Write a program to identify all antonyms and synonyms of a word.

• Introduction
In Natural Language Processing (NLP), understanding synonyms (words with similar
meanings) and antonyms (words with opposite meanings) is essential for various applications,
including sentiment analysis, information retrieval, and text summarization. Synonyms can
help broaden the understanding of a text, while antonyms provide a contrast that can clarify
meanings and enhance descriptions.

• Theory
• Synonyms: Words that have similar meanings. For example, "happy" and "joyful" are
synonyms.
• Antonyms: Words that have opposite meanings. For example, "happy" and "sad" are
antonyms.
To find synonyms and antonyms programmatically, we can use the WordNet lexical database,
which is part of the Natural Language Toolkit (NLTK) in Python. WordNet provides a rich
vocabulary database and allows us to explore relationships between words.

Code:
import nltk
from nltk.corpus import wordnet

# Download the WordNet resource if not already downloaded

nltk.download('wordnet')
nltk.download('omw-1.4')

def find_synonyms_antonyms(word):
"""
This function finds synonyms and antonyms of a given word using WordNet.

Parameters:
word: str - The input word to find synonyms and antonyms for

Returns:
dict - A dictionary containing synonyms and antonyms
"""
synonyms = set()
antonyms = set()

# Get all synsets for the given word

for syn in wordnet.synsets(word):
# Add synonyms
for lemma in syn.lemmas():
synonyms.add(lemma.name()) # Add the synonym
# Check for antonyms
if lemma.antonyms():
antonyms.add(lemma.antonyms()[0].name()) # Add the antonym

26
 3174205 211310142028

return {
'synonyms': list(synonyms),
'antonyms': list(antonyms)
}

# Example usage
if __name__ == "__main__":
word_to_lookup = input("Enter a word to find its synonyms and antonyms: ")
result = find_synonyms_antonyms(word_to_lookup)

print(f"\nSynonyms of '{word_to_lookup}': {result['synonyms']}")

print(f"Antonyms of '{word_to_lookup}': {result['antonyms']}")

Output:

27
 3174205 211310142028

Practical 12
Write a program to find hyponymy, homonymy, polysemy for a given word.

• Introduction
In Natural Language Processing (NLP), understanding the relationships between words is
crucial for tasks such as semantic analysis and language generation. Three important concepts
in this context are:

1. Hyponymy: This is a relationship between words where one word (the hyponym) is a
more specific term than another (the hypernym). For example, "sparrow" is a hyponym
of "bird."
2. Homonymy: This refers to two or more words that sound the same (homophones) or
are spelled the same (homographs) but have different meanings. For example, "bat" can
refer to a flying mammal or a piece of sports equipment.
3. Polysemy: This describes a single word that has multiple meanings. For instance, the
word "bank" can mean a financial institution or the side of a river.

The Natural Language Toolkit (NLTK) library in Python provides access to WordNet, a lexical
database that can be used to explore these relationships. This program demonstrates how to
find hyponyms, hypernyms, and calculate polysemy for given words.

Code:
import nltk
from nltk.corpus import wordnet

# Ensure NLTK resources are downloaded

nltk.download('wordnet')

# Function to find hypernyms

def find_hypernyms(word):
synsets = wordnet.synsets(word)
if synsets:
hypernyms = synsets[0].hypernyms()
return hypernyms
return []

# Function to find hyponyms

def find_hyponyms(word):
synsets = wordnet.synsets(word)
if synsets:
hyponyms = synsets[0].hyponyms()
return hyponyms
return []

# Function to find polysemy

def calculate_average_polysemy():
nouns = set(wordnet.all_synsets('n'))
verbs = set(wordnet.all_synsets('v'))

28
 3174205 211310142028

# Compute average polysemy for nouns

total_noun_polysemy = sum(len(synset.hyponyms()) for synset in nouns)
average_noun_polysemy = total_noun_polysemy / len(nouns)

# Compute average polysemy for verbs

total_verb_polysemy = sum(len(synset.hyponyms()) for synset in verbs)
average_verb_polysemy = total_verb_polysemy / len(verbs)

return average_noun_polysemy, average_verb_polysemy

# Input word for analysis

word = 'apple'
print(f"Finding relationships for the word: {word}")

# Finding hypernyms
hypernyms = find_hypernyms(word)
print(f"Hypernyms of '{word}': {[hypernym.name() for hypernym in hypernyms]}")

# Finding hyponyms
hyponyms = find_hyponyms('fruit')
print(f"Hyponyms of 'fruit': {[hyponym.name() for hyponym in hyponyms]}")

# Calculate average polysemy

average_noun_polysemy, average_verb_polysemy = calculate_average_polysemy()
print(f"Average polysemy of nouns: {average_noun_polysemy:.2f}")
print(f"Average polysemy of verbs: {average_verb_polysemy:.2f}")

Output:

29
 3174205 211310142028

Practical 13
Write a program to calculate the score and polarity of text data using vader
analyzer and TextBlob.

• Introduction

Sentiment analysis is a crucial task in Natural Language Processing (NLP) that involves
determining the emotional tone behind a body of text. This can be particularly useful in various
applications, such as social media monitoring, customer feedback analysis, and market
research.

Two popular libraries for sentiment analysis in Python are VADER (Valence Aware Dictionary
and sEntiment Reasoner) and TextBlob.

• VADER is specifically designed for sentiment analysis in social media texts, such as
tweets. It uses a lexicon of words that are assigned sentiment scores, which makes it
highly effective for short and informal text.

• TextBlob is a more general-purpose NLP library that provides simple API access to
common natural language processing tasks, including sentiment analysis. It uses a
predefined lexicon and the concept of "polarity" and "subjectivity" to gauge sentiment.

Code:
# Install the required libraries if you haven't already
# !pip install vaderSentiment textblob

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

from textblob import TextBlob

# Sample text data for sentiment analysis

text_data = [
"I love the new features in this product! It's amazing.",
"The service was terrible, and I will never return.",
"I'm not sure how I feel about the changes. It's a bit confusing.",
"What a fantastic experience! Highly recommend it to everyone.",
"This is the worst day ever; I'm so upset."
]

# Function to analyze sentiment using VADER

def analyze_sentiment_vader(text):
analyzer = SentimentIntensityAnalyzer()
score = analyzer.polarity_scores(text)
return score

# Function to analyze sentiment using TextBlob

def analyze_sentiment_textblob(text):
blob = TextBlob(text)

30
 3174205 211310142028

return {
'polarity': blob.sentiment.polarity,
'subjectivity': blob.sentiment.subjectivity
}

# Analyze sentiment for each text

for sentence in text_data:
print(f"Text: {sentence}")

# VADER Sentiment Analysis

vader_result = analyze_sentiment_vader(sentence)
print(f"VADER Sentiment Score: {vader_result}")

# TextBlob Sentiment Analysis

textblob_result = analyze_sentiment_textblob(sentence)
print(f"TextBlob Polarity: {textblob_result['polarity']}, Subjectivity: {textblob_result['subjectivity']}")

print("\n" + "="*80 + "\n")

Output:

31
3174205 211310142028

32