Enhancing fraud detection in auto insurance and credit card transactions: a novel approach integrating CNNs and machine learning algorithms

Vehicle insurance fraud detection

Numerous studies have traditionally utilized conventional linear techniques to identify fraudulent insurance claims. However, the application of machine learning techniques in this domain has gained significant popularity. Recent research suggests that leveraging machine learning will play a crucial role in developing more effective fraud detection systems. As a result, several scholars have focused on exploring various techniques to enhance fraud detection performance and tackle the issue of class imbalance. They have evaluated the predictive capabilities of these techniques through comprehensive comparative analyses.

In the study by Wang & Xu (2018), a novel approach was proposed for the detection of auto insurance fraud. The methodology employs a deep learning model integrated with Latent Dirichlet Allocation (LDA)-based text analytics. Deep neural networks are trained to enable accurate fraud detection by extracting text features from accident descriptions in insurance claims using LDA and combining them with traditional numeric features. The experimental results demonstrate that the proposed model achieves the highest accuracy rate of 91.4% compared to SVM and RF algorithms.

The study proposed by Subudhi & Panigrahi (2020) presented a distinctive hybrid approach that integrates various supervised classifier models with a genetic algorithm (GA)-based fuzzy C-means (FCM) clustering technique to identify fraudulent activities in vehicle insurance claims. According to the results, SVM performed better than any other classifier, having the best accuracy of 88.45%, specificity, and 83.21% sensitivity. In a similar vein, Majeed, Abdullah & Mushtaq (2021) enhanced the existing study by proposing a fuzzy clustering technique that synergistically harnesses the capabilities of both the modified whale optimization algorithm (MWOA) and FCM algorithm. The findings showed that the new method generated an accuracy of 96.25%.

Motivated by the need to achieve even higher accuracy rates, researchers explored alternative algorithms and optimization techniques. Salmi & Atif (2021) proposed a new method for detecting fraudulent claims using data mining by utilizing two sample techniques (SMOTE and ROSE) to eliminate the class disparity and test two distinct feature subsets. The results illustrated that random forest performs better than logistic regression with a 95.24% accuracy.

Similarly, Lv et al. (2022) developed a resilient model for auto insurance fraud detection was constructed using a Logistic-SVM approach, and its performance was compared against conventional algorithms, including logistic regression and SVM. The outcome of the evaluation demonstrated that the Logistic-SVM model exhibited superior performance, achieving the highest accuracy rate of 96.1% and outperforming the other algorithms. Further this study by Xia, Zhou & Zhang (2022) introduced a novel deep-learning framework for the identification of auto insurance fraud. The proposed approach integrates CNN, long- and short-term memory (LSTM), and deep neural network (DNN) architectures to leverage their respective strengths. The results demonstrate that the proposed CNN-LSTM model surpasses previous deep learning models, exhibiting improved accuracy, recall rate, and precision with values of 89.6%, 90.7%, and 89.6%, respectively.

Aslam et al. (2022) employed three predictive models, namely naïve Bayes, SVM, and logistic regression, to detect vehicle insurance fraud. Among these models, SVM exhibited the highest accuracy and specificity scores, reaching 94% and 99.77%, respectively. Conversely, the naïve Bayes (NB) algorithm demonstrated the highest precision score of 23.07.

Parallel to these advancements, in this study by Supriya et al. (2023), a hybrid approach is suggested that integrates federated learning (FL), the genetic algorithm (GA), and particle swarm optimization (PSO) to enhance the detection of automobile insurance fraud. The results indicate that the proposed hybrid model achieves the highest accuracy of 94.47%. In a recent study, Abakarim, Lahby & Attioui (2023) introduced an innovative approach to address the challenge of imbalanced data through the utilization of analysis-based techniques. The study incorporates three pre-trained CNN models, namely AlexNet, InceptionV3, and Resnet101, which undergo a streamlined process by removing redundant layers. These CNN models are subsequently integrated in parallel with a novel 1D CNN model using Bagged Ensemble Learning, facilitating an effective solution for imbalanced data classification. In this approach, a SVM classifier is utilized to extract individual predictions from each CNN model. Subsequently, the outcomes of these models are combined using the majority polling technique, resulting in a consolidated prediction. The best performance accuracy for the proposed model was 98%. Correspondingly, A predictive model using machine learning techniques was developed by Okagbue & Oyewole (2023) to improve the fraud detection accuracy of Abakarim, Lahby & Attioui (2023) method and identify potential instances of fraudulent vehicle insurance claims. This study evaluated six algorithms SVM, ANN, AdaBoost, XGboostNB, LR, DT, and RF the best result was 98.5% for Random forest algorithms.

Moreover, Maina, Moso & Gikunda (2023) introduced a new method to deal with imbalanced data by employing the oversampling method and the proposed model is based on the XGBoost algorithm to improve fraud detection efficiency. The findings indicated that XGBoost demonstrates strong performance when applied with SMOTE to address imbalanced training datasets, outperforming other algorithms in the study.

Credit card fraud detection

Recently, credit cards have become a prevalent means of purchasing essential goods that individuals may not be able to afford immediately. However, with the increasing use of credit cards, fraud incidents are also on the rise. Therefore, there is a pressing need to develop a robust and accurate model that can effectively predict and detect credit card fraud (Singh et al., 2023). With the rise of big data and the imbalance of data is a significant issue. The imbalance of data is one of the inevitable problems in business, which will affect the performance of classification models.

Prior research on credit card theft revealed that K-nearest neighbor (KNN) outperformed naïve Bayes (NB), RF, DT, SVM, J48, and binary classification technique (BCT) (Zhao & Guan, 2023).

The harmful impact of imbalanced data structures on various classification models, including decision trees, artificial neural networks (ANNs), and SVMs, has been widely recognized in academic literature (Ren et al., 2023). As a result, a considerable number of researchers have dedicated their efforts to the development of techniques for addressing imbalanced data by employing various methods, which can be categorized into three main approaches: data-level, feature-level, and algorithm-level.

Researchers and practitioners have proposed numerous approaches to address the challenge of identifying fraudulent transactions and overcoming the issue of imbalanced data. Various machine learning techniques have enhanced fraudulent transaction detection accuracy, including supervised and unsupervised methods such as SVM, random forest, isolation forest, local outlier factor, autoencoder, and others. The primary objective is to differentiate between fraudulent and legitimate transactions. Many researchers have extensively examined different approaches and conducted thorough evaluations to determine and compare their predictive prowess. To detect fraudulent activities and address the imbalance data issue, Fiore et al. (2019) introduced a novel voting mechanism based on artificial neural networks and an ensemble model that utilizes deep recurrent neural networks to model sequential data and overcome the unbalanced data. The results obtained from the study showcased the impressive performance of their proposed model, with a precision rate of 97.48% and a recall rate of 77.95%, underscoring its effectiveness.

This performance was improved by Benchaji et al. (2021) by merging the capabilities of three sub-methods. The first sub-method, the uniform manifold approximation and projection (UMAP), was employed to identify the most informative predictive features. The second sub-method involved utilizing long short-term memory (LSTM) networks to incorporate transaction sequences. Lastly, the attention mechanism was utilized to further enhance the performance of the LSTM networks. Similar in vein, Esenogho et al. (2022) presented a new method to improve credit card detection accuracy and also deal with imbalance data by using a long short-term memory (LSTM) method for identifying credit card fraud by employing a neural network ensemble classifier in combination with a hybrid data resampling approach. This new method outperformed sensitivity by 99.6% and specificity by 99.8%. Parallel to these advancements, Sharma, Singh & Chandra (2022) suggested an innovative two-phase oversampling technique that combines knowledge transfer, leveraging the synergistic benefits of the Synthetic Minority Over-sampling Technique (SMOTE) and generative adversarial network (GAN). The results showed the best performance for the SMOTEified technique with a precision of 85% and recall of 80%.

Forough & Momtazi (2021) suggested an ensemble model that utilizes deep recurrent neural networks and artificial neural network-based voting mechanisms to identify fraudulent actions by analyzing data sequences. The results clearly indicate that the suggested model surpasses the current leading models in all evaluation categories. Further, Cheah, Yang & Lee (2023) proposed a new approach to address the issue of class imbalance effectively and increase fraud detection accuracy. This approach is SMOTE+GAN and GANified-SMOTE. The findings reported that the GANified-SMOTE model achieved the highest precision by 94% and recall by 85%. A novel method called Deep Boosting Decision Trees (DBDT) has been proposed for detecting fraud by Xu et al. (2023) which combined the power of gradient boosting and neural networks to enhance fraud detection capabilities. The proposed method outperformed the best performance by AUC 80%. In Lei et al. (2023), the authors introduced a distributed neural network model (DDNN) designed for credit card fraud detection. The model leverages a model optimization algorithm to federate credit card transaction data. The results showed that the accuracy of this model is higher than previous studies by 99.94%.

Moreover, Fanai & Abbasimehr (2023) introduced a two-stage framework to enhance fraud detection accuracy. It employs a deep Autoencoder (AE) for dimensionality reduction and three deep learning classifiers: DNN, RNN, and CNN-RNN. Bayesian optimization optimizes the models’ hyper parameters. The empirical results highlighted the superiority of the proposed approach in enhancing performance. AE-DNN outperformed the baseline DNN model across all evaluation metrics. Additionally, AE-RNN and AE-CNN_RNN surpass their respective baselines, achieving impressive results with an F score of 83%, Recall of 73%, and precision of 97.8%.

Van Belle, Baesens & De Weerdt (2023) introduced CATCHM, an innovative credit card fraud detection method that utilizes representation learning (RL) based on network analysis. This study demonstrates that CATCHM surpasses existing state-of-the-art methods. This outcome substantiates the practical significance of this approach for the industry.

The study by Leevy, Hancock & Khoshgoftaar (2023) conducted an evaluation of binary and one-class classification techniques for credit card fraud detection. Five binary-class classification (BCC) learners and three one-class classification (OCC) learners were assessed using performance metrics such as AUPRC (Area Under Precision-Recall Curve) and AUC (Area Under the ROC Curve). The results indicated that binary classification approaches outperformed one-class classification methods in detecting credit card fraud. The BCC learners achieved AUPRC scores ranging from 85.7% to 74.90%, and CatBoost exhibited the highest AUPRC score of 85.67%.

Zhao & Guan (2023) introduced CTCN, a method for detecting credit card fraud that utilizes a combination of CTGAN and TCN to handle the problem of imbalanced data effectively. This study enhanced CTGAN by including NCL to address the issue of class overlap in imbalanced datasets. The results demonstrated that the proposed approach attained exceptional performance with regards to Recall by 82.99%, F1-Score by 81.87, and AUC-ROC by 91.47%.

On other hand, some studies used CNN and machine learning. For instance, Alharbi et al. (2022) aims to enhance the credit card fraud detection uses a text2IMG conversion technique to transform transaction data into images, and then applies a 16-layer CNN architecture for fraud classification. The text2IMG conversion method proposed in the attached article to represent transaction data as images is a novel aspect of their work. The findings performed that the proposed model achieved the highest accuracy of 99.87%. Moreover, Mizher & Nassif (2023) presented a novel approach for credit card fraud detection using a CNN in conjunction with two machine learning algorithms. The results confirmed that the Random Forest classifier achieved the highest accuracy of 99.78% in detecting credit card fraud on the highly imbalanced dataset.

Even though the existing literature on fraud detection methods, there is a noticeable research gap in comparing the results of traditional methods with more advanced techniques. Most studies only compare their findings with a limited range of traditional methods such as SMOTE, hybrid models of GAN with SMOTE, and deep autoencoder neural networks. By expanding the scope of comparison to include more advanced oversampling techniques and ensemble methods, deeper insights can be gained into the effectiveness of different approaches.

Furthermore, the literature primarily evaluates shallow machine learning models like SVM and logistic regression or explores integrating different neural network architectures such as RNN and LSTM with CNN. However, comparing these approaches with more advanced ensemble methods, such as combining ensemble machine learning algorithms like KNN and DT with a CNN model, could provide a more comprehensive understanding of fraud detection techniques. Moreover, while some studies have utilized CNNs for fraud detection, there is a need to improve the feature extraction techniques to enhance model performance, particularly for complex datasets. Feature extraction assumes a pivotal role in converting raw data into a more suitable format for machine learning algorithms. Exploring sophisticated techniques for feature extraction can bridge this gap and make significant contributions to enhancing the overall performance of fraud detection models.

Moreover, the implementation of sophisticated financial detection methods effectively accomplishes the necessary objective. Nevertheless, machine learning and deep learning approaches have effectively tackled these issues on a significant magnitude in recent years. Therefore, there is a need for enhancement in these techniques with regards to the identification of unusual attack patterns, velocity estimates, and big data analysis. Hence, this study aims to introduce a novel algorithm that enhances the accuracy of fraud detection. This will be achieved by developing a CNN architecture capable of extracting intricate and profound features that indicate fraudulent activities. The proposed approach establishes a robust foundation for analysis by identifying and capturing complex patterns and anomalies. Consequently, the extracted intricate features will be integrated into advanced machine learning models, effectively leveraging the benefits of both deep learning and traditional machine learning techniques.

The importance of the suggested CNN model for fraud detection can be explained by the following observations:

Firstly, CNN models play a crucial role in fraud detection, especially in credit card fraud, because they are highly skilled at automatically extracting spatial and temporal characteristics. This capability allows them to perceive intricate patterns in transaction data that may not be immediately evident to human analysts or conventional machine learning methods. The suggested CNN model utilizes this advantage to improve the accuracy of fraud detection by capturing complex features and subtle patterns that are indicative of fraudulent actions.

Furthermore, CNNs in fraud detection applications have the ability to immediately learn features from raw data, which leads to a reduced need for feature engineering. This feature greatly lowers the need for domain specialists to perform intensive feature engineering, resulting in CNN models that are extremely adaptable to different fraud detection scenarios. The model provided in the study showcases this benefit, illustrating its potential use not just in detecting credit card fraud but also in potentially detecting insurance fraud.

Moreover, the incorporation of CNNs with ensemble machine learning techniques introduces an innovative method that greatly enhances the performance of fraud detection. The suggested model can provide more precise and dependable fraud detection compared to conventional methods by utilizing deep features derived by CNNs. The importance of CNNs in improving the prediction skills of downstream models is emphasized by this integrated approach.

The usefulness of the suggested CNN model is confirmed by its excellent accuracy, precision, recall, and AUC measures, especially in the field of credit card fraud detection. The performance indicators emphasize the model’s dependability and its ability to minimize incorrect identifications of fraud, which is a critical factor in real fraud detection systems where the expense of false alarms can be significant. Lastly, the ability to adapt across several domains: in addition to its effectiveness in detecting credit card fraud, the suggested CNN model’s versatility in detecting other types of fraud, such as insurance fraud, demonstrates its wider usefulness as a generic fraud detection method. The model’s adaptability highlights its importance, providing a strong and adaptable solution that can be customised to address various fraud detection jobs (Khedgaonkar, Singh & Raghuwanshi, 2021; Sindi et al., 2021).

Data description

To investigate the techniques utilized by state-of-the-art models in detecting occurrences of credit card fraud and auto insurance claim fraud, our analysis employs two real-world datasets obtained from Kaggle, namely vehicle insurance (https://www.kaggle.com/datasets/shivamb/vehicle-claim-fraud-detection) and European credit card transactions, for the purpose of fraud detection. The datasets in this study display an inherent class imbalance, with a notable disproportion between the number of fraudulent and non-fraudulent instances (https://www.kaggle.com/mlg-ulb/creditcardfraud/home). The summary statistics for these datasets are presented in Table 1.

Auto insurance and credit card fraud were specifically chosen as the focus areas for this study due to their significant economic impact and prevalence compared to other forms of fraud. Consider the following evidence that underscores the importance of these two domains: first, Auto insurance fraud is a major problem costing the industry billions of dollars annually. The Insurance Information Institute estimates that auto insurance fraud adds $5.6–$7.7 billion in excess payments to auto insurance claims each year in the US alone. This translates to higher premiums for all policyholders. The pervasiveness and substantial financial toll make auto insurance fraud a critical area demanding advanced detection techniques. Second, credit card fraud is one of the most common types of identity theft. According to the Federal Trade Commission’s 2021 Consumer Sentinel Network report, credit card fraud was the most reported type of identity theft, constituting over 390,000 reports. The high volume and direct monetary losses to both consumers and financial institutions underscore the need for robust credit card fraud detection methods.

The techniques used in auto insurance and credit card fraud, such as falsified claims, identity theft, and transaction manipulation, are representative of strategies employed across many other fraud domains. Therefore, developing effective detection models for these two areas can provide insights and methodologies applicable to combating fraud in other sectors as well (https://www.michigan.gov/difs/consumers/fraud/what-is-auto-insurance-fraud). Moreover, From a data perspective, auto insurance and credit card transactions generate vast amounts of structured data suitable for machine learning model development. The availability and quality of data in these domains make them conducive to researching advanced fraud detection techniques like the CNN-based approach proposed in this study (https://www.bankrate.com/insurance/car/fraud/).

Furthermore, the financial services and insurance industries have been at the forefront of adopting machine learning for fraud detection due to the high stakes involved. Focusing on auto insurance and credit card fraud aligns with the sectors most likely to benefit from and implement the novel techniques presented in this research (https://www.chargebackgurus.com/blog/auto-insurance-chargebacks). Lastly, the selection of auto insurance and credit card fraud as the focal points for this study is driven by their substantial economic impact, pervasiveness, representativeness of broader fraud strategies, data availability, and alignment with industries actively seeking advanced detection solutions. By targeting these two critical domains, this research aims to make a significant contribution to the field of fraud detection with potential for wider applicability.

State-of-the-art CNN model

CNNs mimic the human brain’s cerebral cortex’s intricate structure, exhibiting remarkable effectiveness. Their performance relies on a substantial training dataset, facilitating the development of a sophisticated model. Feature learning in this model is accomplished through the utilization of the backpropagation method and the gradient descent optimization algorithm. The system framework of the proposed method is visually presented in Fig. 1. The architectural design you have elucidated corresponds to a classification-oriented CNN. It encompasses diverse layers, including fully connected layers, batch normalization, rectified linear unit (ReLU) activation, dropout, softmax, and classification layers. In the subsequent discussion, we shall delve into the mathematical equations of each layer within this architecture. We first describe the structure of the art CNN model and then describe our proposed approach for credit card and auto insurance fraud detection.

Dropout layer (0.5)

Another crucial component of CNN is the Dropout layer. This layer acts as a mask, reducing the contributions of certain neurons to the subsequent layer while allowing other neurons to function, as depicted in Fig. 2. When a Dropout layer is applied to an input vector, some of its features are diminished, while certain hidden neurons are excluded in the case of a hidden layer. Dropout layers play a vital role in CNN training as they effectively mitigate the risk of overfitting the training data. In our research, Dropout layers with a threshold of 0.25 have been employed.

Feature extraction

Feature extraction initiates with a dataset comprising the primary features and employs them to produce supplementary features that are intended to be informative and devoid of redundancy. This procedure aids in the ability to generalize and potentially enhances the interpretation and performance scores for classification tasks.

Feature extraction, a fundamental preprocessing step in machine learning, involves converting dataset information into a more analytically comprehensible format. Traditionally, this task has relied on human expertise. However, deep learning models have revolutionized this field by automatically extracting features from raw data. Deep learning, especially through CNNs, autonomously processes and learns features from signals or images without human intervention.

In our study, we develop a CNN with a fully connected layer to distinguish between fraudulent and non-fraudulent cases. CNNs comprise multiple layers, including fully connected layers, pooling, and Batch Normalization, which play crucial roles. The fully connected and pooling layers primarily handle feature extraction from the input data. Nevertheless, increasing the number of layers in a network escalates its complexity and the risk of overfitting. To address this concern, our research focuses on training a model that effectively extracts features and classifies samples while maintaining a minimal layer structure to reduce complexity.

High-performance filtering

High-performance filtering (HPF) is a methodology where the two most effective machine learning algorithms are chosen for integration into our ensemble learning framework. The primary goal of this HPF strategy is to identify the most suitable estimators that can be incorporated into the ensemble model to yield the final results. This approach focuses on optimizing the accuracy and consistency of the model’s predictions by utilizing the best-performing algorithms in the ensemble configuration. However, it’s important to note that if suboptimal accuracy is observed in half of the seven machine learning algorithms evaluated, the ensemble models may not surpass the performance of the base learners. To mitigate this risk and enhance robustness, a strategic selection of the top two algorithms. This selective approach not only addresses potential performance issues but also reduces the overall training time. The subsequent section of our study presents a detailed analysis of the feature extraction performance observed during our experiments. Having identified the top-performing machine learning algorithms through High-Performance Filtering, we then integrate these into our ensemble model to enhance overall performance. Specifically, the two most effective algorithms are selected for each dataset and incorporated into an ensemble voting classifier, employing a weighted average approach (soft voting).

Soft voting ensemble learning is an ensemble method that combines the predictions from multiple machine learning models to make a more robust final prediction. It considers the predicted probabilities for each class that each constituent model outputs and computes a weighted average of these probabilities. Specifically, a weighted average of the class probabilities is calculated using preset model weights to account for differences in each model’s expected competence on the problem at hand.

In soft voting, multiple base models such as Support Vector Machines, K-Nearest Neighbors, Decision Trees etc. are trained independently on the same dataset. During prediction, each model generates probability estimates for class membership given a new data instance. For example, in a binary fraud classification application, each model may output the probability that the instance is fraudulent or legitimate. The individual probability outputs from all models are then averaged and weighted to determine the ensemble model’s final probability estimates per class. The ensemble model ultimately classifies the instance into whichever class achieves the highest weighted probability.

Compared to relying on any single model, this soft voting approach allows multiple specialized models to contribute their complementary strengths in making more robust predictions. By aggregating models that are likely to make diverse errors, the ensemble can help cancel out biases, reduce variance, and enhance predictive performance beyond what any individual constituent model can achieve alone. The class with the highest weighted probability average is selected as the final prediction. The ability of ensembles formed by soft voting to make collective predictions leverages the differentiated skills of multiple models, thus improving generalization capacity and reducing sensitivity to noise versus individual models. By combining calibrated probability outputs across models, the soft voting ensemble can produce accurate and stable classifications. This is particularly valuable in applications like fraud detection systems, where the cost of diagnostic errors can be quite high (Akyol et al., 2024)

Ensemble classifier

To construct an efficient model, ensemble models are formed by amalgamating base models. The ensemble model utilizes a combination of diverse learning algorithms to effectively address a classification or regression problem that cannot be adequately resolved by any single model in isolation. In this research, we used soft voting ensemble learning. Initially, we used training data to train basic models such as NB, SVM, DT, and K-NN as shown in Fig. 1. Following training, we used test data to evaluate our models’ performance, with each model making an individual prediction. The ensemble learning approach incorporates the predictions of these models as supplementary inputs, enabling the ensemble model to learn and generate the final prediction collectively.

Data preprocessing

Initially, it was observed that the original datasets for Vehicle Insurance and European credit card transactions exhibited a discrepancy in the number of instances between the fraud and non-fraud categories, as illustrated in Table 1. This imbalance within the database posed challenges for developing an effective predictive model, particularly for the minority category representing fraud in the Vehicle Insurance and European credit card transactions datasets. To address this issue, we employed oversampling technique (Buda, Maki & Mazurowski, 2018), which involved randomly replicating instances from the minority category to match the number of instances in the majority category, as demonstrated in Table 1.

Secondly, the data is categorized into two types: fraud and non-fraud, where the majority represents normal cases and the minority, abnormal. The dataset features are of two kinds: categorical and numerical. Categorical features further include both ordered and unordered types. These datasets have categorical features, some of which are not sequentially ordered. For example, the Vehicle Insurance dataset includes the marital status of users. Since computers only process numerical data, these categorical features must be digitally converted. For non-sequential categorical features, we use One-Hot Encoding. For instance, the gender feature, with two categories, is represented as one-hot vectors: male = (1) and female = (0). However, some features exhibit a sequential order, like day of week, which includes seven categories: Saturday to Friday. For these sequential features, One-Hot Encoding is unsuitable due to the inherent order among categories. Instead, we apply sequential encoding, assigning distinct natural numbers to each category. For instance, the Vehicle Insurance dataset includes the marital status of users. Since computers only process numerical data, these categorical features must be digitally converted. For non-sequential categorical features, we use One-Hot Encoding. For instance, the gender feature, with two categories, is represented as one-hot vectors: male = (1) and female = (0). However, some features exhibit a sequential order, like day of week, which includes seven categories: Saturday to Friday. For these sequential features, One-Hot Encoding is unsuitable due to the inherent order among categories. Instead, we apply sequential encoding, assigning distinct natural numbers to each category. For example, days of the week are mapped as Saturday = (1), Sunday = (2), Monday = (3), and so on through Friday = (7).

Proposed CNN model

The objective of this study is to employ a newly developed CNN model to classify the datasets into two distinct categories: fraud and non-fraud. The architecture of this model comprises a total of 12 layers, encompassing a feature input layer, convolution layers with Rectified Linear Unit (ReLU) activation functions, batch normalization, maximum pooling, two fully connected layers, and a softmax activation layer for the final output. The detailed structure of the model can be illustrated in Fig. 1, while specific parameter values for each layer are provided in Table 2.

Machine learning algorithms

The machine learning algorithms leverage the extracted features from the feature layers of the CNN model to perform classifications. The specific layer of the trained CNN model is selected and employed for feature extraction. our approach utilizes the fc_3 layer from the CNN model, as outlined in Table 1. Subsequently, the classification phase involves the processing of the extracted features from this single layer. For classification purposes, we employ the NB, SVM, DT, and KNN algorithms as classifiers, as depicted in Fig. 1.

Performance metrics

When evaluating the effectiveness of machine learning and deep learning models, various metrics are utilized, with specific learning tasks often requiring particular metrics for emphasis (Shaukat et al., 2020). For this study, we consider the following metrics as essential and provide detailed explanations below.

Accuracy, which is defined as the ratio of correctly predicted instances to the total number of predictions, serves as a fundamental metric for evaluating classification model precision. A higher accuracy value signifies a more precise classification model, making it a desirable outcome (Deng et al., 2016).

Another crucial metric is the F1-score, which harmonizes the precision and recall of a model. This metric proves particularly valuable in scenarios where there is a skewed class distribution, such as a significant discrepancy in sample sizes across classes. F1 score enables a comprehensive assessment of model performance by providing a balanced measure between precision and recall. A higher F1-score indicates superior model performance compared to other models (Shaukat et al., 2020). These metrics are computed as (11)Accuracy = T P + T N T P + T N + F N + F P , (12)p r e c i s i o n = TP TP + FP , (13)Recall = TPR = T P T P + F N (14)F1 − score = TPR = 2 T P 2 T P + F P + F N (15)S p e c i f i c i t y = T N R = T N T N + F P .

True positive (TP) denotes the count of positively labeled samples that the model has accurately identified as positive. Conversely, true negative (TN) represents the quantity of samples correctly classified as negative by the model, referring to instances genuinely belonging to the negative class. False positive (FP) indicates the instances that are incorrectly labeled as positive by the model, despite being negative. on the other hand, false negative (FN) is the count of positive samples erroneously classified as negative (Shaukat et al., 2020).

Performance evaluation for auto insurance fraud dataset

Initially, the auto insurance fraud oversampled dataset was divided into three subsets: 70% for training, 15% for validation, and 15% for testing purposes. The proposed CNN model for the Vehicle Insurance Fraud dataset was trained over 300 epochs, using a mini-batch size of 15 and a total of 47,400 iterations, which took approximately 27 min and 50 s. The Adam solver was employed with an initial learning rate of 0.00001 for optimization. Figure 3 presents the training and validation graphs, showing the progression of loss values and accuracy for the proposed CNN model. During training, the model achieved an accuracy of 60% with a corresponding loss of 0.7, while the validation accuracy was also 60% with a loss of 0.7%. Specific hyperparameters used for training the CNN model, such as learning rate, batch size, number of epochs, etc. This will allow reproducibility of our results.

The performance metrics obtained from the CNN model are presented in Table 2 and Fig. 4, both in tabular format and as a visual graphical bar chart. These results indicate that the CNN model did not accurately classify the Vehicle Insurance Fraud dataset, likely due to the limited amount of data available. Deep learning models generally require a larger dataset to achieve better performance.

Table 3 and Fig. 5 present the performance evaluation of seven machine learning (ML) techniques used to classify features extracted from the CNN model. The results demonstrate the accurate classification of features achieved by the ML techniques. Among the methods examined, Knn1 emerged as the most effective ML technique, achieving a high accuracy of 96.62%. This was accomplished by extracting features from the performance metrics of the Vehicle Insurance Fraud dataset using the CNN model and subsequently classifying them with Knn1. To further enhance the classification accuracy, a soft voting ensemble-based approach was implemented for the final detection. The CNN-ensemble ML model’s robustness was evaluated using confusion matrices for each dataset, considering metrics such as accuracy, precision, sensitivity, F1-score, and AUC. In the context of the finance sector, maximizing sensitivity is crucial as it ensures timely and accurate detection of fraudulent digital payments.

Figure 6 displays the confusion matrix for the CNN-ensemble ML architecture applied to the Vehicle Insurance Fraud dataset, which demonstrated a lower misclassification The misclassification errors for Non-Fraud and Fraud were found to be 0 and 66, respectively. The accuracy values of the DT, Knn5, Knn3, Knn1, GSVM, LSVM, NB, and ensemble models were determined to be 96.53%, 90.91%, 93.38%, 96.62%, 69.98%, 68.75%, 68.29%, and 98.86%, respectively, as presented in Table 3, showcasing the classification performance metrics of all the ML models.

In ML and DL, the AUC-ROC curve holds significant importance in evaluating the performance of classification tasks, particularly in the context of imbalanced datasets. This curve serves as a valuable metric for assessing the predictive accuracy of a model. The ROC curve represents the probability of correct classification for different classes, with the X-axis representing the False Positive Rate (FPR) and the Y-axis representing the True Positive Rate (TPR).

Regarding the CNN-ensemble ML architectures, Fig. 7 illustrates the ROC curves and corresponding AUC values for different CNN-ensemble ML configurations. The CNN-ensemble ML model proposed in this study exhibits an impressive AUC value of 96.5%, indicating its exceptional predictive precision for the Vehicle Insurance Fraud dataset.

Comparison with previous studies for auto insurance fraud dataset

This section demonstrates the superior performance of our ensemble model in comparison to previous endeavors within the same domain. Table 4 provides a comparative analysis of our proposed approach against existing methodologies in classifying the Vehicle Insurance Fraud dataset.

Several studies have investigated deep learning approaches for auto insurance fraud detection, focusing on imbalanced datasets. Xu et al. (2023) developed Deep Boosted Decision Trees utilizing gradient boosting and neural networks, with a specific emphasis on maximizing AUC performance. They evaluated an auto insurance fraud dataset but did not provide comprehensive accuracy or other statistical measures. More relevantly, Xia, Zhou & Zhang (2022) proposed a CNN-LSTM model for auto insurance fraud detection, aiming to reduce the complexity of expert-driven feature engineering. By automatically learning feature representations, their approach achieved an accuracy of 89.6% and precision of 90.7% on a standard insurance dataset.

The first column of the table lists prior studies, while the second column outlines the methodologies employed in each study, which could entail either the introduction of a novel model by the authors or the application of cutting-edge models. The third column provides details about the datasets utilized in each study. To facilitate a comprehensive assessment and identification of the most effective strategies, the table highlights the best method and the highest achievement for each study.

According to Table 4, our approach combining CNN with ensemble machine learning techniques (KNN1 and DT) outperforms some of the state-of-the-art models. These methods excel in extracting more robust and distinct deep features, leading to optimal classification outcomes. Additionally, to address the dataset’s imbalance, an oversampling technique is employed during network training. Despite the utilization of different datasets, our proposed CNN consistently delivers superior performance compared to other studies that incorporate dual classification to tackle the similar issue.

Performance evaluation for European credit card transactions fraud dataset

The proposed CNN model for the European credit card transactions Fraud dataset utilized an oversampled dataset. The dataset was divided into 70% for training, 15% for validation, and 15% for testing. The model was trained for 300 epochs using a mini-batch size of 15, resulting in 129,900 iterations. The training process took approximately 312 min and 29 s. The Adam solver was employed using an initial learning rate of 0.00001. Figure 8 presents the training and validation graphs, showcasing the loss values and accuracy of the proposed CNN model. The training accuracy reached 94.54% with a loss of 0.2, while the validation accuracy also achieved 94.54% with a loss of 0.2%. The performance metrics obtained from the CNN model are presented in Table 5 and visualizes in Fig. 9, providing a comprehensive overview of the model’s effectiveness in accurately classifying European credit card transactions Fraud. Deep learning algorithms, such as CNN, benefit from large amounts of training data, and the European credit card transactions Fraud dataset contains substantial data suitable for such purposes. Furthermore, Table 6 and Fig. 10 present the performance of seven machine learning techniques employed to classify the features of the CNN model, offering a comparative analysis.

The results demonstrate that the ML techniques accurately classified the features extracted from each CNN model. Among the methods, Knn1 emerged as the strongest ML technique for classification, achieving the highest accuracy of 99.97%. This was accomplished by extracting features from the performance metrics of the European credit card transactions Fraud dataset using the CNN model and classifying them with Knn1. Furthermore, Fig. 11 presents the confusion matrix for the CNN-ensemble ML architecture applied to the European credit card transactions Fraud dataset, which exhibited a lower misclassification error rate. The misclassification errors for Non-Fraud and Fraud were 0 and 15, respectively. The accuracy values for the DT, Knn5, Knn3, Knn1, GSVM, LSVM, NB, and ensemble models were 99.97, 99.93, 99.95, 99.97, 94.64, 94.92, 94.50, and 99.99, respectively, as shown in Table 5, which presents the classification performance measures for all ML models.

Regarding the CNN-ensemble ML architectures, Fig. 12 display the ROC curves and corresponding AUC values for the CNN-ensemble ML configurations. The CNN-ensemble ML model, as the proposed model, demonstrates an AUC value of 100%, indicating its exceptional predictive precision for European credit card transactions.

Comparison with previous studies for European credit card transactions fraud dataset

This section showcases the superior performance of our ensemble model compared to previous endeavors in the field. Table 7 presents a comparative analysis of our proposed approach in classifying the European credit card transactions Fraud dataset against existing methodologies.

Several studies have explored machine learning techniques for credit card fraud detection. For instance, Lei et al. (2023) utilized distributed deep neural networks and achieve a remarkable accuracy of 99.94%. Xu et al. (2023) proposed Deep Boosted Decision Trees that combine neural networks and gradient boosting, with a specific focus on optimizing the AUC metric and attaining an AUC of 81.24%. Sharma, Singh & Chandra (2022) design a CNN-based architecture incorporating GANs and SMOTE for oversampling, achieving an 85% precision; however, they do not evaluate AUC. More closely aligned with our proposed approach, Fanai & Abbasimehr (2023) develop a two-stage model involving an autoencoder for dimensionality reduction followed by deep learning classifiers. Their best-performing configuration, employing CNN and RNN, achieves an F1- score of 83.72%.

To further enhance the identification of credit card fraud in imbalanced datasets, we propose an integrated framework that combines a CNN-based feature extractor with ensemble machine learning classifiers. Our model incorporates targeted oversampling to balance the class distributions, enabling the classifiers to better discern the characteristics of the minority positive class. We conduct a comprehensive empirical evaluation using stratified sampling and metrics such as precision, recall, and AUC, specifically designed for handling data skewness. The objective of our proposed system is to achieve nearly 100% precision, ensuring reliable fraud alerts, while simultaneously maximizing the detection rate. Through the utilization of deep feature representations and intelligent fusion of multiple classifiers, our technique outperforms previous benchmarks in terms of accuracy, F1-score, and AUC.