2 ways we’re using AI:
1. Suggesting Messages (Text Generation)
2. Content Moderation (Sentiment Analysis & NLP)
Suggesting Messages
Context Awareness
Natural Language Processing (NLP) models like BERT and GPT enhance
understanding of context to reduce false positives. Steps:
● Preprocessing: Tokenize and clean the text. Convert text into subword
tokens if using BERT/GPT.
● Model Loading: Load a pre trained BERT or GPT model fine-tuned for
harmful content detection.
● Embedding Generation: Convert text into contextual embeddings that
capture semantic meaning.
● Contextual Understanding: Leverage attention mechanisms to understand
relationships between words and phrases in the text.
● Prediction: Pass embeddings through classification layers to determine
harmfulness.
● Fine-Tuning: Update the model using labeled examples specific to the
application.
● Deployment: Integrate the model into the messaging system for live analysis.
Also for suggesting messages part, keep in mind:
● Transformer Architecture: Both models rely on self-attention mechanisms, which
allow them to capture long-range dependencies in text efficiently. This makes the
models adept at understanding the context of a conversation or a user prompt.
● Autoregressive Decoding: In message suggestion tasks, the models generate
tokens sequentially. The generation of each token depends on the previously
generated tokens and the input prompt. This ensures the output is coherent and
contextually appropriate.
● Prompt Conditioning: The models are conditioned with a well-crafted prompt (e.g.,
“Generate three open-ended questions”). This guides the generation process to align
the output with the desired structure and intent, such as creating engaging social
media messages.
● Tokenization: Before processing, input text is divided into smaller units called
tokens. The model processes these tokens to predict the next token iteratively,
forming a complete response.
● Beam Search or Sampling: For diverse and creative outputs, methods like Top-k
Sampling or Nucleus Sampling are often employed. These introduce randomness by
selecting from the top-ranked tokens at each step, ensuring variability in generated
messages while maintaining relevance.
Content Moderation
Sentimental Analysis
While sentiment analysis typically categorizes text as positive, negative, or neutral, it
can be adapted to detect offensive language by associating specific negative sentiments or
aggressive tones with harmful content. The model might be fine-tuned on datasets where
aggressive or hateful speech is labeled as "negative sentiment," triggering the detection of
offensive language.
● Lexicon-Based Approaches: In these approaches, a predefined dictionary (or
lexicon) of words associated with specific sentiments (e.g., positive, negative,
neutral) is used. Sentiment scores are calculated based on the presence of these
words in the text. Examples include:
○ VADER (Valence Aware Dictionary and sEntiment Reasoner): It is a
lexicon and rule-based sentiment analysis tool specifically tailored for social
media text. VADER detects not just positive or negative sentiment but also the
intensity and polarity of emotions, which can be useful in detecting offensive
or aggressive content.
○ SentiWordNet: A lexical resource for sentiment analysis that assigns
sentiment scores to words. This can be used to identify negative emotions or
offensive language.
● Rule-Based Sentiment Analysis: Rules are created to identify sentiment based on
patterns in text. For example, the presence of certain words (e.g., "hate," "violence,"
"abuse") might trigger a "negative" sentiment label. This approach can be useful in
detecting offensive or hateful language.
Offensive language blocking:
● Binary Classification: This is the most common technique where the model is
trained to classify text as either "offensive" or "non-offensive." The training data
consists of labeled examples, where each text instance is tagged with one of these
two labels. Popular algorithms for text classification include:
○ Logistic Regression
○ Naive Bayes
○ Support Vector Machines (SVM)
○ Neural Networks (especially deep learning models like CNNs and RNNs)
● LLM(Large Language Model) and LSTM(Long Short Term Memory):
○ LLMs like GPT (Generative Pre-trained Transformer) or BERT
(Bidirectional Encoder Representations from Transformers) are
pre-trained on massive text corpora and can be fine-tuned for specific tasks
such as harmful content detection. These models excel in understanding
context, nuances, and semantics.
○ LSTMs are a type of Recurrent Neural Network (RNN) designed to handle
sequential data effectively, such as sentences. They remember long-term
dependencies, making them suitable for analyzing the sequence and context
of words in a message.
