Llama2中文大模型——牛刀小试

Llama2中文大模型——牛刀小试

前言

• Meta开源的Llama从第一版开始，效果就很不错，有很多开源LLM都是基于它训练的，例如Vicunna、Alpaca

• 在23年7月，Meta又开源了Llama的第二版 Llama2，分为Pretrian、Chat版本，包含7B、13B、70B模型

  类别	模型名称	下载地址	备注
  Pretrian	Llama2-7B	huggingface下载	基础版
  Pretrian	Llama2-13B	huggingface下载	基础版
  Pretrian	Llama2-70B	huggingface下载	基础版
  Chat	Llama2-7B-Chat	huggingface下载	聊天对话增强
  Chat	Llama2-13B-Chat	huggingface下载	聊天对话增强
  Chat	Llama2-70B-Chat	huggingface下载	聊天对话增强

• 大量团队再次基于Llama2进行升级或开发新的模型，都有非常好的效果，如

  • Microsoft 的 WizardLM-70B
  • NVIDIA 的 Llama2-70B-SteerLM-Chat
  • LMSYS 的 Vicuna-13B

• 在中文场景下，由于原Llama2的中文预训练数据较少（仅占0.13%），中文能力较弱，所以我们在构建中文问答场景的应用时，往往需要提升模型的中文能力，一般可以采用微调和预训练两种方式：

  • 预训练：效果更好，但需要大规模高质量的中文数据、大规模的算力资源
  • 微调：速度快，需要的资源少，效果不错

• 对于普通个人或小型企业来说，大规模文本的预训练较难以承受，通常会采用微调的方式。

• 不过，国内有一款基于Llama2做预训练的开源中文大模型 Atom，针对中文做了多方面的强化工作，效果好、开源、免费、可商业化。我们完全可以基于该训练好的中文模型，再做特定行业的数据微调，实现商业场景化的中文大模型。

• 如果再结合LangChain，便可以更方便地基于Llama2开发文档检索、问答机器人和智能体应用等（官方已集成LangChain）

• 下面就来小试一下该模型 FlagAlpha/Atom-7B

• 需要注意的是：为了能方便快速地使用模型，不必纠结于环境配置、安装，我采用了Kaggle

  • 公开的Notebook地址：https://www.kaggle.com/code/alionsss/flagalpha-atom-7b/notebook

更新库

• Kaggle上的accelerate库需要更新一下，同时还要安装最新的bitsandbytes库（用于对模型进行量化，加载模型需要用到）
• 注：如果不使用Kaggle，你应当根据官方的requirements.txt文件安装依赖

pip install --upgrade accelerate
pip install bitsandbytes

• 输出信息样例

Requirement already satisfied: accelerate in /opt/conda/lib/python3.10/site-packages (0.25.0)
Collecting accelerate
  Obtaining dependency information for accelerate from https://files.pythonhosted.org/packages/a6/b9/44623bdb05595481107153182e7f4b9f2ef9d3b674938ad13842054dcbd8/accelerate-0.26.1-py3-none-any.whl.metadata
  Downloading accelerate-0.26.1-py3-none-any.whl.metadata (18 kB)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from accelerate) (1.24.3)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from accelerate) (21.3)
Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate) (5.9.3)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from accelerate) (6.0.1)
Requirement already satisfied: torch>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from accelerate) (2.0.0)
Requirement already satisfied: huggingface-hub in /opt/conda/lib/python3.10/site-packages (from accelerate) (0.20.2)
Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.10/site-packages (from accelerate) (0.4.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->accelerate) (3.0.9)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.12.2)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (4.5.0)
Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (1.12)
Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.1)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (3.1.2)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub->accelerate) (2023.12.2)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from huggingface-hub->accelerate) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub->accelerate) (4.66.1)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (1.26.15)
Requ