sklearn 绘制多分类ROC曲线

最新推荐文章于 2024-04-18 11:11:32 发布

向阳争渡

最新推荐文章于 2024-04-18 11:11:32 发布

阅读量5.4k

点赞数

CC 4.0 BY-SA版权

分类专栏：机器学习文章标签： sklearn AUC

本文链接：https://blog.csdn.net/yangyang_yangqi/article/details/91970430

机器学习专栏收录该内容

10 篇文章

订阅专栏

本文介绍了一个自然语言理解(NLU)模型的评估流程，包括精度、召回率的计算，以及使用ROC-AUC曲线衡量模型在多分类任务上的表现。通过解析测试数据集中的意图识别结果，展示了如何利用sklearn库进行模型性能评估。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

from rasa_nlu.model import Interpreter
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn import metrics
from sklearn.preprocessing import OneHotEncoder
import json
import time
import numpy as np
import pandas as pd
import matplotlib
matplotlib.use('Agg')
import os
from matplotlib import pyplot as plt 
def get_path(root):
    paths = []
    if os.path.isdir(root):
        lists = os.listdir(root)
        for i in lists:
            paths.append(os.path.join(root,i))
    return paths
root = '/app/split_data/test'
paths = get_path(root)
examples = []
for i in paths:
    print(i)
    f = open(i,'r')
    data = json.load(f)
    examples.append(data['rasa_nlu_data']['common_examples'])
    f.close()
interpreter2 = Interpreter.load("./models/nlu/ECS-nlu/models_shuffle/")

result = []
intent = []
ranking = []
for i in examples:
    for item in i:
        rasa_result= interpreter2.parse(item['text'])
        result.append(rasa_result['intent']['name'])
        intent.append(item['intent'])
        ranking.append(rasa_result['intent_ranking'])
'''
for k,v in ecs_test_data.items():
    rasa_res = interpreter2.parse(k)
    result.append(rasa_res['intent']['name'])
    intent.append(v)
    ranking.append(rasa_result['intent_ranking'])
'''
acc_score = precision_score(intent,result,average='weighted')
print('accuracy score:{}'.format(acc_score))
rec_score = recall_score(intent,result,average='weighted')
print('recall score:{}'.format(rec_score))
intent_lists = set(intent)
raw = {}
arrays = []
for i in intent_lists:
    raw[i] = 0
for j in ranking:
    tmp = raw.copy()
    for k in j:
         tmp[k['name']] = k['confidence']
    confs = list(tmp.values())
    arrays.append(confs)
arrays = np.array(arrays)
intent = np.array(intent).T
label = pd.DataFrame(intent,columns = ['label'])
unique = label.ix[:,'label'].unique()
for j in range(len(unique)):
    label.ix[:,'label'] = label.ix[:,'label'].apply(lambda x:j if x == unique[j] else x)
enc = OneHotEncoder()
enc.fit(label)
label_one_hot = enc.transform(label).toarray()
print(metrics.roc_auc_score(label_one_hot,arrays,average='weighted'))