An Ensemble Learning Hybrid Recommendation System Using
Content-Based, Collaborative Filtering, Supervised Learning and Boosting
Algorithms
Dataset - MovieLens 1M (https://dl.acm.org/doi/10.1145/2827872)
MSLE - 2.3
RMSLE - 1.5
Methods Used in the Hybrid Recommendation System:
1. Collaborative Filtering with SVD:
○ Purpose: To identify patterns in user-item interactions and make
recommendations.
○ How it works:
■ Uses Singular Value Decomposition (SVD) to decompose the
user-item interaction matrix into three matrices:
■ R=U⋅S⋅VTR = U \cdot S \cdot V^T
■ R: User-Item Matrix
■ U: User preferences
■ S: Singular values (importance of features)
■ V: Item attributes
■ Reduces data dimensionality and uncovers hidden relationships.
■ Low-rank approximation of the original matrix is used to predict
missing ratings.
2. Content-Based Filtering with TF-IDF and Cosine Similarity:
○ Purpose: To recommend items similar to those a user liked before,
based on content features.
○ How it works:
■ Extracts features (e.g., genre, director, cast) using TF-IDF (Term
Frequency-Inverse Document Frequency).
■ Represents each item as a vector of feature weights.
■ Measures similarity between items using Cosine Similarity.
■ Recommends items with high similarity scores.
3. SVM Classifier:
○ Purpose: To enhance the recommendation accuracy using
classification.
○ How it works:
■ Trains on content-based features obtained from TF-IDF.
■ Support Vector Machine (SVM) finds the best decision
boundary (hyperplane) to classify items.
■ Learns complex patterns in high-dimensional data for precise
recommendations.
4. Ensemble Learning Algorithms:
○ Purpose: To combine multiple models for improved accuracy and
robustness.
○ How it works:
■ Decision Tree (DT): Splits data based on features to make
decisions.
■ Random Forest (RF): Combines multiple decision trees to
enhance accuracy and reduce overfitting.
■ Support Vector Regression (SVR): Predicts continuous ratings
by learning from user-item interactions.
■ Each model's strength is leveraged, and weaknesses are
mitigated by combining them.
5. Boosting Algorithms:
○ Purpose: To improve model performance by iteratively correcting
errors.
○ How it works:
■ XGBoost:
■ Builds an ensemble of decision trees.
■ Uses Gradient Boosting to minimize prediction errors by
learning from previous mistakes.
■ Efficiently handles sparse data and complex feature
relationships.
■ CatBoost:
■ Specializes in high-cardinality categorical data.
■ Handles missing values automatically.
■ Captures intricate connections between categorical
variables.
6. Evaluation Metrics:
○ Used to measure the performance of the recommendation system:
■ RMSE (Root Mean Square Error)
■ MAE (Mean Absolute Error)
■ MAPE (Mean Absolute Percentage Error)
■ MSLE (Mean Square Logarithmic Error)
■ RMSLE (Root Mean Square Logarithmic Error)
7. Ensemble Approach:
○ Combines DT, RF, and SVR to leverage:
■ DT’s interpretability and non-linear pattern capturing.
■ RF’s robustness and reduced overfitting.
■ SVR’s ability to minimize prediction errors in high-dimensional
data.
○ Extracts complex patterns and provides more accurate and
personalized recommendations.
A hybrid collaborative filtering mechanism for product recommendation
system
Accuracy - 97.7%
Lowest error rate - 0.02%
Major parameters like RMSE, MAE, precision, Accuracy, recall, F-measure and error
rate.
Book Dataset - available from author request
Methodology of the Proposed CbDNCF Design (in Simple Terms)
The proposed CbDNCF (Chimp-based Deep Neural Collaborative Filtering)
scheme uses the Chimp Algorithm for optimization and a Deep Neural Network
(DNN) for predicting user ratings. It is designed in five main phases:
1. Data Training and Preprocessing Phase
● Objective: Clean the data to remove noise and prepare it for the model.
● Steps:
○ Import the raw book rating data.
○ Filter out noise using a Chimp initialization function. (subset of
features, threshold, smoothing parameters) (candidate -> parameters
to tune, different values)
○ Minimize error rates to ensure accurate training.
○ Equation Used:
■ Error Calculation: Er = \frac{1}{2n} \sum (Brd - ni)^2 Where:
■ BrdBrd = Book rating data
■ nini = Noise present in the data
■ ErEr = Error after noise filtering
○ The cleaned data is then passed to the next phase for classification.
2. Feature Extraction Phase
● Objective: Select relevant features to reduce complexity and improve
prediction accuracy.
● Steps:
○ Extract the most relevant features from the user rating data.
○ Ignore meaningless or redundant features.
○ Equation Used:
■ Feature Extraction:D* = Rf(j) - Ef(j) Where:
■ D∗ = Extracted features
■ Rf = Matched rating features
■ Ef = External or meaningless features
■ j = Iteration count
3. Classification Phase
● Objective: Classify products based on user ratings to recommend the best
ones.
● Steps:
○ Calculate the average rating for each product.
○ Classify products as either High Rating (above 7) or Low Rating (7 or
below).
○ Equations Used:
■ Rating Calculation: Rating(pred)= \frac{\sum Brd}{n}
Classification Condition:
if (Brd > 7) then High Rating
else Low Rating
■
4. Parameter Tuning Phase
● Objective: Optimize the DNN model's parameters for better performance.
● Steps:
○ Utilize the Chimp Algorithm to tune hyperparameters in the DNN.
○ Adjust weights and biases for optimal predictions.
○ Continue iteration until the best solution is obtained.
5. Output Layer Phase
● Objective: Generate the final prediction and recommendations.
● Steps:
○ Compile the processed data and predictions.
○ Display the top-rated products to the user.
○ Validate the results by evaluating performance metrics.
Additional Information:
● The value 7 is chosen as a threshold based on the Chimp Algorithm's
optimal solution, representing the best iteration count for finding high-rated
products.
● The CbDNCF model is implemented in Python and is compatible with all
operating systems.
Summary:
The proposed CbDNCF model leverages a combination of:
● Chimp Algorithm: For optimizing DNN parameters.
● Deep Neural Network: For learning complex patterns in user ratings.
● Collaborative Filtering: For recommending top-rated products.
This approach ensures accurate predictions, reduced complexity, and better
recommendation outcomes.
It initializes a set of candidate solutions, each representing different combinations
of parameters for noise filtering.
These parameters might include:
● Threshold values for filtering noisy data points.
● Smoothing factors to minimize fluctuations in the data.
By exploring different parameter values, the Chimp Algorithm identifies the optimal
settings that minimize noise while preserving meaningful patterns.
It initializes a set of candidate solutions, each representing different combinations
of parameters for noise filtering.
These parameters might include:
Threshold values for filtering noisy data points.
Smoothing factors to minimize fluctuations in the data.
By exploring different parameter values, the Chimp Algorithm identifies the optimal
settings that minimize noise while preserving meaningful patterns.
Sentiment analysis based distributed recommendation system (2024)
Dataset - Amazon Review Dataset (Electric and Musical Items Category)
Precision - 89.1%
Recall - 84.1%
- JSON Format
- User Reviews converted to numerical value, i.e., sentiment score
# ReviewText -> tokenized -> stop word removal -> tf-idf vectors -> logistic
regression -> sentimental rating from 1 to 5
- Reviews filtered based on up-votes and helpfulness score
- Overall effect - does downvotes exceed upvotes
- Normalized Rating -> overall rating + sentimental rating + overall effect
- Distributed ALS using Pyspark
- Hyperparameter tuning + cross validation
Social media reviews based hotel recommendation system using
collaborative filtering and big data
Dataset - have used four yelp datasets in.json file format for the dataset, each
containing more than 3 lakh rows. The datasets are sorted according to the type of
trip planned ex.- Busi ness, Leisure, etc. The link for the dataset is shared below:
https://drive.google.com/drive/
folders/1K0LWTpeLwKBqlSIexfqf_CPYG8nJSYPT?usp=sharing
- addressing issues like cold-start (lack of user data for new users/items) and
scalability (system performance as the amount of data grows).
The proposed methodology of the paper can be summarized as follows:
1. Dataset Preparation with Filtration:
○ Process raw data using HTML tags and eliminate unnecessary words
via extractive summarization.
○ Normalize words using a Porter Stemmer to standardize them for
further analysis.
2. Effective Dictionary Creation:
○ Construct a vocabulary of relevant keywords that are used to evaluate
and categorize terms.
○ The vocabulary is used to assess the sentiment and relevance of each
term.
3. Sentiment Analysis & Weighting Feedback:
○ Perform sentiment analysis to determine the emotional tone of user
reviews.
○ Hotels are ranked based on the highest sentiment scores.
4. Information Retrieval for Long Comments:
○ Long comments are processed to remove extraneous content like
punctuation.
○ Term Frequency (TF) and phrase gravity are calculated to identify the
most important terms in reviews.
5. Recommendation System (Emotion & Content-Based Filtering):
○ Keywords from user feedback (both short and long comments) are
stored in a thesaurus.
○ Sentiment classification is used to determine ratings for hotels that
match a user’s preferences.
○ Matrix Factorization (ALS) is used to filter and predict user ratings,
helping the system recommend the best hotels.
6. Ratings/Reviews Imputation:
○ User Profile Construction: Build a user profile based on features they
care about in hotels (e.g., amenities).
○ Similarity Calculation: Calculate similarity between users based on
their preferences for hotel features using cosine similarity.
7. User Covariance Matrices & Score Prediction:
○ Calculate predicted ratings for hotels using a formula based on user
similarity.
○ The system predicts scores for each hotel based on similarities
between users' profiles and preferences.
8. Capsule Networks (CapsNet):
○ CapsNet is used to improve the recommendation system by
preserving detailed information about the user’s preferences.
○ The architecture includes three layers:
■ Convolutional Layer: Extracts features from input data.
■ Primary Capsule Layer: Captures multi-dimensional features
(instantiation parameters).
■ Digit Capsule Layer: Outputs predictions based on the
extracted features.
In simple terms, the methodology combines data processing, sentiment analysis,
and matrix factorization with the advanced Capsule Networks to accurately
recommend hotels by considering user preferences and feedback.
